-- Logs begin at Thu 2022-05-26 17:19:16 UTC, end at Thu 2022-05-26 18:39:38 UTC. -- May 26 18:02:13.669881 dv-cinder-843493-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 26 18:02:13.681530 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 26 18:02:13.684064 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 26 18:02:13 2022] *** May 26 18:02:13.684064 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 26 18:02:13.684064 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 26 18:02:13.684064 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: nodename: dv-cinder-843493-1-iscsi May 26 18:02:13.684064 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: machine: x86_64 May 26 18:02:13.684064 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: clock source: unix May 26 18:02:13.684064 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: pcre jit disabled May 26 18:02:13.684064 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: detected number of CPU cores: 4 May 26 18:02:13.684064 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: current working directory: / May 26 18:02:13.684064 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: detected binary path: /usr/bin/uwsgi-core May 26 18:02:13.685165 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: your processes number limit is 47680 May 26 18:02:13.685165 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: your memory page size is 4096 bytes May 26 18:02:13.685165 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: detected max file descriptor number: 2048 May 26 18:02:13.685165 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: lock engine: pthread robust mutexes May 26 18:02:13.685165 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: thunder lock: enabled May 26 18:02:13.685165 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 26 18:02:13.685165 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 26 18:02:13.740157 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: Python main interpreter initialized at 0x56422e90f9c0 May 26 18:02:13.740157 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: python threads support enabled May 26 18:02:13.740157 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: your server socket listen backlog is limited to 100 connections May 26 18:02:13.740157 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: your mercy for graceful operations on workers is 90 seconds May 26 18:02:13.740537 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: mapped 403077 bytes (393 KB) for 2 cores May 26 18:02:13.740608 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: *** Operational MODE: preforking *** May 26 18:02:13.740894 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: *** uWSGI is running in multiple interpreter mode *** May 26 18:02:13.742316 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: spawned uWSGI master process (pid: 80314) May 26 18:02:13.742316 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: spawned uWSGI worker 1 (pid: 80324, cores: 1) May 26 18:02:13.742316 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: spawned uWSGI worker 2 (pid: 80325, cores: 1) May 26 18:02:13.742316 dv-cinder-843493-1-iscsi devstack@keystone.service[80314]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 26 18:02:13.741121 dv-cinder-843493-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 26 18:02:19.533011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 18:02:19.533011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: from cryptography.utils import int_from_bytes May 26 18:02:19.533917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 18:02:19.533917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: from cryptography.utils import int_from_bytes May 26 18:02:19.583961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 18:02:19.584131 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 18:02:19.584270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] command line args: [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 18:02:19.584518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 18:02:19.584642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 18:02:19.584642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: from cryptography.utils import int_from_bytes May 26 18:02:19.585353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ================================================================================ {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 18:02:19.585353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] admin_token = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.585481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 18:02:19.585481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: from cryptography.utils import int_from_bytes May 26 18:02:19.585579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] config_dir = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.585579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.585579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] config_source = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.585768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] debug = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.585967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.587076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.587076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.587076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.587076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.587334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.587334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_config_append = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.587511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.587676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_dir = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.587866 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_file = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.588088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.588339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.588598 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.588836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.589025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.589216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.589436 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.589690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.589952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.590169 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.590395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.590627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.590878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.591171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.591458 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.591891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.592139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] publish_errors = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.592420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.592761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.593078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.593359 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.593651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.593898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] standard_threads = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.594076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.594263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.594442 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.594619 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] use_journal = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.594793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] use_json = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.594974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] use_stderr = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.595150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] use_syslog = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.595379 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.595641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.595875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.596136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.596373 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.596618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.596856 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.597100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.597333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.597577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.597847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.598100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.598335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.598580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.598813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.599056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.599296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.599531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.599783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.600033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.600278 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.600511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.600762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.601020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.601269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.601518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.601774 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.602005 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.602233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.602463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.602710 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.602954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.603186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.603414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.603656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.603900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.604135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.604380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.604666 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.604848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.605077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.605305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.605533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.605780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.606011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.606245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.606488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.606717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.606951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.607179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.607411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.607644 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.607888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.608116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.609001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.609264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.609467 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.609872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.610122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.610354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.610603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.610838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.611077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.611307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.611568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.611803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.612033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.612269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.612503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.612772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.613006 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.613251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.613495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.613754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.613980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.614211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.614456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.614693 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.614926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.615323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.615459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.615630 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.615861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.616094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.616325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.616588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.616821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.617058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.617296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.617531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.617857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.618096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.618331 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.618558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.618790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.619017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.619249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.619476 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.619724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.619956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.620201 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.620435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.620688 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.620944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.621183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.621412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.external = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.621641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.621958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.622187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.622416 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.password = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.622670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.token = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.622880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.623112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.623354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.623583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.623859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.624702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.625009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.625813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.625813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.625813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.626152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.626271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.626455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.626690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.626941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.627189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.627425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.627677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.627954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.628201 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.628462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.628698 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.628947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.629179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.629414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.629655 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.629923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.630155 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.630395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.630628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.630878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.631123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.631388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.631623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.631867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.632099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.633079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.633079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.633079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.633079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.633456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.633574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.633793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.634111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.634351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 18:02:19.634495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.634619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 18:02:19.634735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.634847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] command line args: [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 18:02:19.634958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.635075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 18:02:19.635075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ================================================================================ {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 18:02:19.635344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.635344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.635661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] admin_token = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.635828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.636109 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] config_dir = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.636109 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.636451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.636624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] config_source = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.636781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.636964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] debug = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.637155 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.637328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.637328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.637705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.637912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.638075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.638255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.638425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.638600 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.638770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.638939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.639111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.639276 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_config_append = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.639485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.639669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.639669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_dir = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.639979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.640132 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_file = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.640240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.640336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.640438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.640557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.640670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.640783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.640893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.641004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.641128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.641242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.641355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.641466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.641578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.641697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.641844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.641957 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.642069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.642227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.642398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.642570 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.642688 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.642688 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.642900 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.643029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.643200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.643490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.643667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.643810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.643926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.644037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.644148 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.644269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.644269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.644489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.644602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.644721 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] publish_errors = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.644835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.644947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.645059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.645179 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.645291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.645410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.645532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.645657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.645827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.645942 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.646055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.646167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] standard_threads = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.646295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.646488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.646649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.646767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.646767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.647086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.647256 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] use_journal = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.647437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.647613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] use_json = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.647790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.647964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] use_stderr = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.647964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] use_syslog = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.648459 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 18:02:19.648633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.648752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.648865 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.648976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.649095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.649218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.649332 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.649444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.649555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.649667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.649831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.649949 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.650089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.650213 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.650328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.650441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.650627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.650775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.650899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.651013 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.651126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.651238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.651350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.651462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.651581 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.651581 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.651893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.651893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.652220 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.652395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.652577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.652577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.652902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.653074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.653245 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.653417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.653607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.653836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.654010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.654183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.654352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.654522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.654706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.654706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.655020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.655273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.655454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.655454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.655827 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.655979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.656095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.656209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.656321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.656435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.656557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.656679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.656794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.656907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.657019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.657132 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.657252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.657367 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.657479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.657593 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.657705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.657705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.657953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.658068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.658189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.658304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.658417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.658528 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.658649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.658769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.658883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.658995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.659107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.659465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.659651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.659769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.659882 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.659994 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.660107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.660229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.660354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] role.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.660468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.660581 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] role.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.660694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.660806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.660918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.661039 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.661153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.661266 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.661379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.661491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.661615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.661757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.661874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.661986 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.662106 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.662223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.662348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.662463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.662575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.662687 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.662800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.662912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.663032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.663147 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.663258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.663371 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.663484 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.663595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.663715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.663830 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.663941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.663941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.664151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.664263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.664386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.664500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.664612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.664724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.664836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.664949 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.665073 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.665195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.665309 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.665421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.665533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.665645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.665786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.665903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.666016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.666128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.666248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.666363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.666482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.666596 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.666708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.666820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.666937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.667051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.667170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.667288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.667402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.667513 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.667624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.667736 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.667855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.667970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.668083 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.668195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.668307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.668419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.668545 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.668660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.668772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.668884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.668995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.669108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.669228 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.669342 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.669454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.669566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.669677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.669844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.669970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.670085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.670199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.670311 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.670423 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.670535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.671169 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.671293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.671406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.671502 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.671595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.671690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.671792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.671889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.671984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.672077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.672189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.672301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.672420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.672535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.672648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.672763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.672875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.672986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.673105 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.connection = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.673228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.673351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.673473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.673587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.673700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.673837 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.673959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.674073 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.674185 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.674298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.674417 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.674530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.674650 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.674765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.674877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.674989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.675101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.external = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.675213 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.675333 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.675448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.675561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.675674 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.675786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.675898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.676021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.password = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.676135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.676248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.token = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.676360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.676473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.676585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.676706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.676820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.676932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677156 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 18:02:19.677618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.677618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.678568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.678568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.678568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.678568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.678568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.678568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.678568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.678568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.678568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.679558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.679558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.679558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.679558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.679558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.679558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.679558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.679558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.679558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.680467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.680467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.680467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.680467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.680467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.680467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.680467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.680467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.680467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.681372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.681372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.681372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.681372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.681825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.681825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.682035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.682146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.682314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.682643 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.682942 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.683189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.683462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.683700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.683938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.684169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.684407 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.684710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.684990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.685235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.685475 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.685741 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.685979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.686225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.686741 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.686937 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.687203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.687448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.687682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.687924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.688154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.688396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.688634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.688883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.689123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.689362 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.689611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.689885 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.690118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.690349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.690596 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.690840 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.691074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.691308 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.691542 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.691773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.692020 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.692278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.692510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.692754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.692998 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.693431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.693562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.693676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.693940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.694185 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.694423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.694775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.695025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.695273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.695518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.695763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.696008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.696238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.696485 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.696846 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.696976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.697181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.697419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.697665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.697935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.698174 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.698428 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.698663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.698906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.699142 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.699376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.699612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.699854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.700099 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.700325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.700570 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.700808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.701056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.701305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.701540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.701815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.702586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.703314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.703314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.703314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.703605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.703738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] role.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.703857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.703971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] role.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.704076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.704182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.704287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.704408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.704633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.704876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.705121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.705364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.705608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.705874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.706119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.706350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.706592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.706841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.707080 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.707322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.707750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.707985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.708234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.708479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.708724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.708975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.709207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.709460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.709711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.709973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.710205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.710435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.710666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.710916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.711166 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.711411 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.711645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.711880 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.712107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.712339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.712574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.712790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.712913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.713021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.713277 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.713521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.713881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.714122 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.714365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.714610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.714853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.715083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.715323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.715565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.715812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.716039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.716279 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.716582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.716842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.connection = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.717087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.717332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.717685 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.718977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.718977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.718977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.719288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.719393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.719627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.719864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.720108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.720344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.720592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.720827 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.721071 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.721298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.721539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.721792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 18:02:19.721965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 18:02:19.742177 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.742402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.747038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.747267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.747491 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.757755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.776948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.777251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.796993 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.816645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.849310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.849517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.855006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.855244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.916156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.916371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.916528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.916661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.916789 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.916914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.917038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.917165 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.917298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.917414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.917535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.917654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.941484 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.941683 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.941852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.942045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.942167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.942295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.942417 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.942558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.942672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.942793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.942913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.943032 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.943165 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.943280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.954892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.955012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.968114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.968332 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.968475 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.968613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.968751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.968751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.969091 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.969244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.969244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.969497 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.969630 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.969816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.970219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.970537 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 18:02:19.987063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:19.987271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 18:02:19.987608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:19.987712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:19.987795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 18:02:19.987928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 18:02:19.988064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:19.988148 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:19.988320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:19.988413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 18:02:19.988575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:19.988657 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:19.988829 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:19.988911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:19.989085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:19.989339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.000633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.000911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.001177 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.001458 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.001839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.002118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.002384 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.002664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.009670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.010817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.013004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.013287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.013568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.013876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.014125 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.014241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.014402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.014694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.014820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.014936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.015069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.015320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.015437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.015600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.015716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.015880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.016035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.016150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.016267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.016495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.016769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.017037 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.043586 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.044240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.045216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.045609 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.046552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.046833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.051806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.052085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.052208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.052350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.052470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.052714 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.059715 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.059996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.060133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.060312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.064824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.064951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.065110 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.065226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.065376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.065490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.065803 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.066022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.066155 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.066290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.066408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.066530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.066651 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.066773 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.066912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.067027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.085481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.085768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.085888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.086052 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.090451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.090763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.094742 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.094994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.095116 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.095290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.100107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.100416 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.103805 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.107621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.107741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.107918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.108036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.108267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.108957 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.109118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.109238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.109499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.109621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.109812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.109932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.129275 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.129560 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.129679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.129897 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.130008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.130133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.130254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.130419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.138189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.138421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.140440 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.140712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.141002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.141292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.143829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.144110 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.144391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.144508 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.144671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.144787 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.144930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.145046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.145210 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.145477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.158462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.158743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.159024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.159582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.159701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.159862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.159979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.160137 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.160255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.160429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.160561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.160762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.161047 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.161327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.178607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.179579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.182060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.182919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.185392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.186256 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.187661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.188432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.189317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.189611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.189950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.190074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.190360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.190647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.197329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.198104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.198992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.199712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.202858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.203148 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.203440 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.203708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.203826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.204017 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.204362 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.204585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.212876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.213163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.213441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.213826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.214118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.214477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.221721 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.222019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.222297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.222497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 18:02:20.222617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.222780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.222898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.223057 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.223175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.223336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.223452 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.223599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.223704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.223873 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.224002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.224137 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.224313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.224428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.224548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.224668 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.224787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.224915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.225028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.225149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.225267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.225412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.225674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.226120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 18:02:20.265384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56422e90f9c0 pid: 80325 (default app) May 26 18:02:20.266095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56422e90f9c0 pid: 80324 (default app) May 26 18:02:20.268750 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830bc29-e290-4af2-ac95-8c6c502090b2 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:20.269041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830bc29-e290-4af2-ac95-8c6c502090b2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:20.269317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830bc29-e290-4af2-ac95-8c6c502090b2 None None] PATH_INFO: `/v3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:20.273652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Thu May 26 18:02:20 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:28.916694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6565438-7278-4e18-b4db-68ba7fab3440 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:28.917286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6565438-7278-4e18-b4db-68ba7fab3440 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:28.917286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6565438-7278-4e18-b4db-68ba7fab3440 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:28.922284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:28.928044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b51e2-fb89-4d44-af40-cf736124ada8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:28.928501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b51e2-fb89-4d44-af40-cf736124ada8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:28.928679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b51e2-fb89-4d44-af40-cf736124ada8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:29.003901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: INFO dbcounter [None req-db6b51e2-fb89-4d44-af40-cf736124ada8 None None] Registered counter for database keystone May 26 18:02:29.025818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG oslo_db.sqlalchemy.engines [None req-db6b51e2-fb89-4d44-af40-cf736124ada8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80325) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 18:02:29.032855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writer thread running {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 18:02:29.210915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG passlib.handlers.bcrypt [None req-db6b51e2-fb89-4d44-af40-cf736124ada8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80325) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 18:02:29.211242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG passlib.handlers.bcrypt [None req-db6b51e2-fb89-4d44-af40-cf736124ada8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80325) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 18:02:29.267785 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-db6b51e2-fb89-4d44-af40-cf736124ada8 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:29.269402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-db6b51e2-fb89-4d44-af40-cf736124ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:29.397210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 26 18:02:29.553053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6565438-7278-4e18-b4db-68ba7fab3440 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:29.621673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: INFO dbcounter [None req-a6565438-7278-4e18-b4db-68ba7fab3440 None None] Registered counter for database keystone May 26 18:02:29.642700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG oslo_db.sqlalchemy.engines [None req-a6565438-7278-4e18-b4db-68ba7fab3440 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80324) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 18:02:29.648970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writer thread running {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 18:02:29.780953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6565438-7278-4e18-b4db-68ba7fab3440 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:29.785547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2f5aa-c346-426e-be34-c846bbfb5691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:29.786241 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2f5aa-c346-426e-be34-c846bbfb5691 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:29.786527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2f5aa-c346-426e-be34-c846bbfb5691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:29.786808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2f5aa-c346-426e-be34-c846bbfb5691 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:29.796999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2f5aa-c346-426e-be34-c846bbfb5691 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:29.855035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: WARNING keystone.common.rbac_enforcer.enforcer [None req-64b2f5aa-c346-426e-be34-c846bbfb5691 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 18:02:29.855299 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2f5aa-c346-426e-be34-c846bbfb5691 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:29.865923 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Thu May 26 18:02:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:02:29.871977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b51e2-fb89-4d44-af40-cf736124ada8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:29.886783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b51e2-fb89-4d44-af40-cf736124ada8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:29.891919 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21668ee-2a5b-4ace-80e2-5016daa5c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:29.893057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21668ee-2a5b-4ace-80e2-5016daa5c71e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:29.893338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21668ee-2a5b-4ace-80e2-5016daa5c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:29.893607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21668ee-2a5b-4ace-80e2-5016daa5c71e admin admin] PATH_INFO: `/v3/projects/bbbdb1cacacf4ba5a41a00d6d478efbe` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:29.895153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21668ee-2a5b-4ace-80e2-5016daa5c71e admin admin] RBAC: Authorizing `identity:get_project(project_id=bbbdb1cacacf4ba5a41a00d6d478efbe)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:30.058313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b21668ee-2a5b-4ace-80e2-5016daa5c71e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 18:02:30.059002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21668ee-2a5b-4ace-80e2-5016daa5c71e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:30.063747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Thu May 26 18:02:29 2022] GET /identity/v3/projects/bbbdb1cacacf4ba5a41a00d6d478efbe => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:32.288452 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d513bd5-3635-4638-9520-3b44853bd1a9 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:32.288993 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d513bd5-3635-4638-9520-3b44853bd1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:32.288993 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d513bd5-3635-4638-9520-3b44853bd1a9 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:32.293121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:02:32.299235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6182c2-2581-4b1e-8811-9398243b18b0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:32.299654 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6182c2-2581-4b1e-8811-9398243b18b0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:32.299830 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6182c2-2581-4b1e-8811-9398243b18b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:32.319775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5b6182c2-2581-4b1e-8811-9398243b18b0 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:32.321318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5b6182c2-2581-4b1e-8811-9398243b18b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:32.330092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 18:02:32.353231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d513bd5-3635-4638-9520-3b44853bd1a9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:32.365349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d513bd5-3635-4638-9520-3b44853bd1a9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:32.369491 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2da36-a658-4e40-857c-fa96d589a3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:32.370155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2da36-a658-4e40-857c-fa96d589a3db admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:32.370441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2da36-a658-4e40-857c-fa96d589a3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:32.370713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2da36-a658-4e40-857c-fa96d589a3db admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:32.378419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2da36-a658-4e40-857c-fa96d589a3db admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:32.380192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2da36-a658-4e40-857c-fa96d589a3db admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:32.388860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Thu May 26 18:02:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:02:32.395265 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6182c2-2581-4b1e-8811-9398243b18b0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:32.404211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6182c2-2581-4b1e-8811-9398243b18b0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:32.409772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795352ef-18fc-4c04-a139-5a07491aa8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:32.410626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795352ef-18fc-4c04-a139-5a07491aa8cf admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:32.410904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795352ef-18fc-4c04-a139-5a07491aa8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:32.411176 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795352ef-18fc-4c04-a139-5a07491aa8cf admin admin] PATH_INFO: `/v3/users/3b98d845867d491c91aeca1e12cf3ace` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:32.412566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795352ef-18fc-4c04-a139-5a07491aa8cf admin admin] RBAC: Authorizing `identity:get_user(user_id=3b98d845867d491c91aeca1e12cf3ace)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:32.414424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795352ef-18fc-4c04-a139-5a07491aa8cf admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:32.415977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Thu May 26 18:02:32 2022] GET /identity/v3/users/3b98d845867d491c91aeca1e12cf3ace => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:34.754024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb71713-adcc-4965-a92a-5e19642b0349 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:34.754565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb71713-adcc-4965-a92a-5e19642b0349 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:34.754565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb71713-adcc-4965-a92a-5e19642b0349 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:34.755939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:34.761187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b44a1-b504-4410-89b2-b8e82b65efa3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:34.761620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b44a1-b504-4410-89b2-b8e82b65efa3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:34.761781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b44a1-b504-4410-89b2-b8e82b65efa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:34.778780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8c8b44a1-b504-4410-89b2-b8e82b65efa3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:34.779992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8c8b44a1-b504-4410-89b2-b8e82b65efa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:34.819026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a2278-d55a-4dc5-b688-7aaf22f801f2 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:34.819292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a2278-d55a-4dc5-b688-7aaf22f801f2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:34.819577 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a2278-d55a-4dc5-b688-7aaf22f801f2 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:34.820918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:02:34.827850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0357552-c328-474c-84f0-20e9499cb7a5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:34.828271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0357552-c328-474c-84f0-20e9499cb7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:34.828656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0357552-c328-474c-84f0-20e9499cb7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:34.871633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:34.873012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG passlib.handlers.bcrypt [None req-a0357552-c328-474c-84f0-20e9499cb7a5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80324) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 18:02:34.873506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG passlib.handlers.bcrypt [None req-a0357552-c328-474c-84f0-20e9499cb7a5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80324) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 18:02:34.894505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b44a1-b504-4410-89b2-b8e82b65efa3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:34.921918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a0357552-c328-474c-84f0-20e9499cb7a5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:34.923047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:34.923594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a0357552-c328-474c-84f0-20e9499cb7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:35.008801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b44a1-b504-4410-89b2-b8e82b65efa3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:35.025114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:02:35.043640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764733b8-0747-4bf9-bd16-a83951e51aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:35.044441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764733b8-0747-4bf9-bd16-a83951e51aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:35.044730 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764733b8-0747-4bf9-bd16-a83951e51aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:35.045012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764733b8-0747-4bf9-bd16-a83951e51aa0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:35.049303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0357552-c328-474c-84f0-20e9499cb7a5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:35.058907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764733b8-0747-4bf9-bd16-a83951e51aa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-764733b8-0747-4bf9-bd16-a83951e51aa0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:35.070824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:35.071832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:02:35.072656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 7/14] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:02:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:02:35.078019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764733b8-0747-4bf9-bd16-a83951e51aa0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:35.155594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0357552-c328-474c-84f0-20e9499cb7a5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:35.158108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:35.176678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:35.195663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764733b8-0747-4bf9-bd16-a83951e51aa0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:35.205065 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1708a-c66a-4c79-9349-e470879dcf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:35.206012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1708a-c66a-4c79-9349-e470879dcf27 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:35.206298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1708a-c66a-4c79-9349-e470879dcf27 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:35.206580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1708a-c66a-4c79-9349-e470879dcf27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:35.237297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8637e9-268b-409f-a352-34abf20754bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:35.237998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8637e9-268b-409f-a352-34abf20754bb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:35.238280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8637e9-268b-409f-a352-34abf20754bb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:35.238550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8637e9-268b-409f-a352-34abf20754bb None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:35.239296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8637e9-268b-409f-a352-34abf20754bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:35.239601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8637e9-268b-409f-a352-34abf20754bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:35.241011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8637e9-268b-409f-a352-34abf20754bb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:35.252655 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 8/15] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:02:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:35.258331 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8637e9-268b-409f-a352-34abf20754bb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:35.287318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1708a-c66a-4c79-9349-e470879dcf27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:35.288875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1708a-c66a-4c79-9349-e470879dcf27 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:35.362241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8637e9-268b-409f-a352-34abf20754bb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:35.386537 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 8/16] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 18:02:35.393005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1708a-c66a-4c79-9349-e470879dcf27 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:35.395788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd685494-81ee-471f-b413-81dc4c900d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:35.395967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:35.396710 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd685494-81ee-471f-b413-81dc4c900d33 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:35.396949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd685494-81ee-471f-b413-81dc4c900d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:35.397223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd685494-81ee-471f-b413-81dc4c900d33 None admin] PATH_INFO: `/v3/users/3b98d845867d491c91aeca1e12cf3ace` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:35.398466 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd685494-81ee-471f-b413-81dc4c900d33 None admin] RBAC: Authorizing `identity:get_user(user_id=3b98d845867d491c91aeca1e12cf3ace)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:35.399945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd685494-81ee-471f-b413-81dc4c900d33 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:35.401561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 9/17] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:35 2022] GET /identity/v3/users/3b98d845867d491c91aeca1e12cf3ace => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:02:35.407589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd685494-81ee-471f-b413-81dc4c900d33 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:35.423162 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:35.495162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1708a-c66a-4c79-9349-e470879dcf27 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:35.514580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd685494-81ee-471f-b413-81dc4c900d33 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:35.529110 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd630d64-57e0-4739-9c12-ef0b8842509a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:35.530639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd630d64-57e0-4739-9c12-ef0b8842509a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:35.530932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd630d64-57e0-4739-9c12-ef0b8842509a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:35.531205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd630d64-57e0-4739-9c12-ef0b8842509a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:35.549080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882590f9-6501-47f4-b203-d8da10f19ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:35.549837 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882590f9-6501-47f4-b203-d8da10f19ced None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:35.549938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd630d64-57e0-4739-9c12-ef0b8842509a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:35.550124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882590f9-6501-47f4-b203-d8da10f19ced None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:35.550400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882590f9-6501-47f4-b203-d8da10f19ced None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:35.551339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd630d64-57e0-4739-9c12-ef0b8842509a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:35.551616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882590f9-6501-47f4-b203-d8da10f19ced None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:35.553080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882590f9-6501-47f4-b203-d8da10f19ced None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:35.554704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 10/18] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:02:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:02:35.560342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882590f9-6501-47f4-b203-d8da10f19ced None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-dd630d64-57e0-4739-9c12-ef0b8842509a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:02:35.572902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:35.573934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:35.574728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:02:35.574728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:02:35.574728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:02:35.574728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:02:35.574728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 9/19] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:35.580608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd630d64-57e0-4739-9c12-ef0b8842509a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:35.635236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:35.642791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:35.662366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882590f9-6501-47f4-b203-d8da10f19ced None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:35.680109 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd630d64-57e0-4739-9c12-ef0b8842509a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:35.699297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcb395-8169-4c6b-994a-e36a3ba76392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:35.699952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcb395-8169-4c6b-994a-e36a3ba76392 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:35.700239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcb395-8169-4c6b-994a-e36a3ba76392 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:35.700595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcb395-8169-4c6b-994a-e36a3ba76392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:35.701424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcb395-8169-4c6b-994a-e36a3ba76392 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.domain.id=default, user.id=3b98d845867d491c91aeca1e12cf3ace) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:35.702429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcb395-8169-4c6b-994a-e36a3ba76392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:35.703916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcb395-8169-4c6b-994a-e36a3ba76392 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:35.710531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 11/20] 10.222.0.46 () {56 vars in 1322 bytes} [Thu May 26 18:02:35 2022] GET /identity/v3/role_assignments?user.id=3b98d845867d491c91aeca1e12cf3ace&scope.domain.id=default&role.id=584beecb022b422f820e77f7623ada5e => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:02:35.714894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a17b8a-3652-453b-9d74-b0b47ba15a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:35.715567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a17b8a-3652-453b-9d74-b0b47ba15a6f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:35.715851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a17b8a-3652-453b-9d74-b0b47ba15a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:35.716126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a17b8a-3652-453b-9d74-b0b47ba15a6f None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:35.716864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a17b8a-3652-453b-9d74-b0b47ba15a6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:35.717236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a17b8a-3652-453b-9d74-b0b47ba15a6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:35.718722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a17b8a-3652-453b-9d74-b0b47ba15a6f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:35.740002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 10/21] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:02:38.299902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd35865-6496-4e35-b41f-ecea172e8fe8 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:38.300429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd35865-6496-4e35-b41f-ecea172e8fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:38.300429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd35865-6496-4e35-b41f-ecea172e8fe8 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:38.301794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 12/22] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:38.307525 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666c822-0570-40b2-ab0a-8a231fd2d523 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:38.307959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666c822-0570-40b2-ab0a-8a231fd2d523 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:38.308510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666c822-0570-40b2-ab0a-8a231fd2d523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:38.423253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d55bcc-6cf6-469f-a3a8-12225dd908af None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:38.423571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d55bcc-6cf6-469f-a3a8-12225dd908af None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:38.423808 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d55bcc-6cf6-469f-a3a8-12225dd908af None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:38.425414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 13/23] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:38.443536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b67fc1-6e23-42fd-8d7d-4668a77fb529 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:38.443845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b67fc1-6e23-42fd-8d7d-4668a77fb529 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:38.444089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b67fc1-6e23-42fd-8d7d-4668a77fb529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:38.472036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-30b67fc1-6e23-42fd-8d7d-4668a77fb529 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:38.473257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-30b67fc1-6e23-42fd-8d7d-4668a77fb529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:38.490017 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9666c822-0570-40b2-ab0a-8a231fd2d523 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:38.491198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9666c822-0570-40b2-ab0a-8a231fd2d523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:38.625198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 11/24] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:38.630806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967d675-a7f4-4692-8682-8c77680fe4a0 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:38.631342 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967d675-a7f4-4692-8682-8c77680fe4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:38.631825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967d675-a7f4-4692-8682-8c77680fe4a0 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:38.633325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 12/25] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:02:38.638211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404747a-025d-44cb-bd87-5c27db6aa4ed None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:38.638842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404747a-025d-44cb-bd87-5c27db6aa4ed None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:38.639871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404747a-025d-44cb-bd87-5c27db6aa4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:38.658500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c404747a-025d-44cb-bd87-5c27db6aa4ed None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:38.660253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c404747a-025d-44cb-bd87-5c27db6aa4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:38.698801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 14/26] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:38.705320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b67fc1-6e23-42fd-8d7d-4668a77fb529 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:38.730378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:38.775556 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 13/27] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:38.776785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404747a-025d-44cb-bd87-5c27db6aa4ed None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:38.886288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:38.942252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404747a-025d-44cb-bd87-5c27db6aa4ed None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:38.956591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b67fc1-6e23-42fd-8d7d-4668a77fb529 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:38.985899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc1fe2-7b2d-4a31-8a5a-b70bd284d360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:38.987287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc1fe2-7b2d-4a31-8a5a-b70bd284d360 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:38.988408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc1fe2-7b2d-4a31-8a5a-b70bd284d360 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:38.988876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc1fe2-7b2d-4a31-8a5a-b70bd284d360 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:38.990086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc1fe2-7b2d-4a31-8a5a-b70bd284d360 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:38.992427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc1fe2-7b2d-4a31-8a5a-b70bd284d360 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:39.028687 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53343f7d-3af9-46e4-a7eb-181aab2990ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:39.031197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53343f7d-3af9-46e4-a7eb-181aab2990ae None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:39.032405 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53343f7d-3af9-46e4-a7eb-181aab2990ae None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:39.032691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53343f7d-3af9-46e4-a7eb-181aab2990ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:39.052222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53343f7d-3af9-46e4-a7eb-181aab2990ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-53343f7d-3af9-46e4-a7eb-181aab2990ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:39.060047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:39.061548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 15/28] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:02:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:39.062480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53343f7d-3af9-46e4-a7eb-181aab2990ae None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:39.147472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 14/29] 10.222.0.46 () {60 vars in 1131 bytes} [Thu May 26 18:02:38 2022] POST /identity/v3/roles => generated 219 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:02:39.148459 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc1fe2-7b2d-4a31-8a5a-b70bd284d360 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:39.181773 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53343f7d-3af9-46e4-a7eb-181aab2990ae None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:39.206571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:39.233191 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b84fb-1ca9-472f-a7bb-7487fe0f9ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:39.234288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b84fb-1ca9-472f-a7bb-7487fe0f9ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:39.234611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b84fb-1ca9-472f-a7bb-7487fe0f9ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:39.234886 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b84fb-1ca9-472f-a7bb-7487fe0f9ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:39.237241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:39.250485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b84fb-1ca9-472f-a7bb-7487fe0f9ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:39.252164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b84fb-1ca9-472f-a7bb-7487fe0f9ba7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-294b84fb-1ca9-472f-a7bb-7487fe0f9ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:02:39.281759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:39.282631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:39.283455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:02:39.283455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:02:39.283455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:02:39.283455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:02:39.283455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 16/30] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:39.283455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0708a-a0f1-40c9-bd11-6b771709e2b7 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:39.283455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0708a-a0f1-40c9-bd11-6b771709e2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:39.283455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0708a-a0f1-40c9-bd11-6b771709e2b7 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:39.283455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 17/31] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:39.285645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0708a-a0f1-40c9-bd11-6b771709e2b7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:39.335647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc1fe2-7b2d-4a31-8a5a-b70bd284d360 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:39.436283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d9fd1-1fd3-4243-9a6d-0d9ee5ca3618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:39.436898 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d9fd1-1fd3-4243-9a6d-0d9ee5ca3618 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:39.437186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d9fd1-1fd3-4243-9a6d-0d9ee5ca3618 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:39.437464 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d9fd1-1fd3-4243-9a6d-0d9ee5ca3618 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:39.438279 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d9fd1-1fd3-4243-9a6d-0d9ee5ca3618 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:39.438592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d9fd1-1fd3-4243-9a6d-0d9ee5ca3618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:39.440126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d9fd1-1fd3-4243-9a6d-0d9ee5ca3618 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:39.455579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0708a-a0f1-40c9-bd11-6b771709e2b7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:39.456798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 15/32] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:02:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:39.458978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d99e0-b4dc-48e6-9b9d-8ea3e9cd85db None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:39.459272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d99e0-b4dc-48e6-9b9d-8ea3e9cd85db None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:39.459552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d99e0-b4dc-48e6-9b9d-8ea3e9cd85db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:39.493368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-395d99e0-b4dc-48e6-9b9d-8ea3e9cd85db None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:39.496005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-395d99e0-b4dc-48e6-9b9d-8ea3e9cd85db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:39.497780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ad95e-8b64-4154-b01e-2b259805e72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:39.499450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ad95e-8b64-4154-b01e-2b259805e72e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:39.499915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ad95e-8b64-4154-b01e-2b259805e72e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:39.500350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ad95e-8b64-4154-b01e-2b259805e72e None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:39.504717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ad95e-8b64-4154-b01e-2b259805e72e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:39.505257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ad95e-8b64-4154-b01e-2b259805e72e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:39.508347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ad95e-8b64-4154-b01e-2b259805e72e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:39.528556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 18/33] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:39.530151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ad95e-8b64-4154-b01e-2b259805e72e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:39.594720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:39.600201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 16/34] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:39.601342 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d99e0-b4dc-48e6-9b9d-8ea3e9cd85db None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:39.652580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:39.675772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ad95e-8b64-4154-b01e-2b259805e72e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:39.721838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eb32b-ed39-48ba-aea8-f38688488a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:39.721838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eb32b-ed39-48ba-aea8-f38688488a3d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:39.721838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eb32b-ed39-48ba-aea8-f38688488a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:39.721838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eb32b-ed39-48ba-aea8-f38688488a3d None admin] PATH_INFO: `/v3/users/3b98d845867d491c91aeca1e12cf3ace` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:39.723615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345eb32b-ed39-48ba-aea8-f38688488a3d None admin] RBAC: Authorizing `identity:get_user(user_id=3b98d845867d491c91aeca1e12cf3ace)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:39.725364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345eb32b-ed39-48ba-aea8-f38688488a3d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:39.729165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 19/35] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:39 2022] GET /identity/v3/users/3b98d845867d491c91aeca1e12cf3ace => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:39.729962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eb32b-ed39-48ba-aea8-f38688488a3d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:39.814339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d99e0-b4dc-48e6-9b9d-8ea3e9cd85db None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:39.866388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eb32b-ed39-48ba-aea8-f38688488a3d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:39.893783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdcdfc-7162-496c-a62d-f2c3f7b8d825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:39.895114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdcdfc-7162-496c-a62d-f2c3f7b8d825 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:39.895781 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdcdfc-7162-496c-a62d-f2c3f7b8d825 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:39.896069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdcdfc-7162-496c-a62d-f2c3f7b8d825 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:39.901757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdcdfc-7162-496c-a62d-f2c3f7b8d825 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:39.901930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:39.902770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdcdfc-7162-496c-a62d-f2c3f7b8d825 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:39.953758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977649d-00e7-4996-8b7c-df74c51a5ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:39.953758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977649d-00e7-4996-8b7c-df74c51a5ad6 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:39.953758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977649d-00e7-4996-8b7c-df74c51a5ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:39.953758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977649d-00e7-4996-8b7c-df74c51a5ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:39.954153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977649d-00e7-4996-8b7c-df74c51a5ad6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:39.955902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977649d-00e7-4996-8b7c-df74c51a5ad6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:39.960717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 17/36] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:02:39 2022] POST /identity/v3/projects => generated 301 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:02:39.962357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdcdfc-7162-496c-a62d-f2c3f7b8d825 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:40.078685 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:40.103947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdcdfc-7162-496c-a62d-f2c3f7b8d825 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:40.131763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 20/37] 10.222.0.46 () {60 vars in 1131 bytes} [Thu May 26 18:02:39 2022] POST /identity/v3/roles => generated 225 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:02:40.133362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39124b-c009-4f6d-8212-7421c73c5ddc None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:40.133653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39124b-c009-4f6d-8212-7421c73c5ddc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:40.133969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39124b-c009-4f6d-8212-7421c73c5ddc None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:40.135539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 21/38] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:40.142208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab127b1-2a0a-4a8f-8438-1b3cde51225b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:40.142527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab127b1-2a0a-4a8f-8438-1b3cde51225b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:40.142770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab127b1-2a0a-4a8f-8438-1b3cde51225b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:40.160769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09427d-21ee-4bbb-a1aa-042e0888ee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:40.161597 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09427d-21ee-4bbb-a1aa-042e0888ee10 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:40.161928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09427d-21ee-4bbb-a1aa-042e0888ee10 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:40.162217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09427d-21ee-4bbb-a1aa-042e0888ee10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:40.164345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09427d-21ee-4bbb-a1aa-042e0888ee10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:40.166106 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09427d-21ee-4bbb-a1aa-042e0888ee10 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:40.167924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 18/39] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:02:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:40.174970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f30ac-fb5c-43c9-affa-220797e4a67f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:40.175385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f30ac-fb5c-43c9-affa-220797e4a67f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:40.175672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f30ac-fb5c-43c9-affa-220797e4a67f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:40.176238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2ab127b1-2a0a-4a8f-8438-1b3cde51225b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:40.177118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 19/40] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:40.177886 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2ab127b1-2a0a-4a8f-8438-1b3cde51225b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:40.178420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f30ac-fb5c-43c9-affa-220797e4a67f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:40.278677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 22/41] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:02:40.280086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b13fa-de3c-409f-a5d8-da8dfbb2c7f3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:40.280536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b13fa-de3c-409f-a5d8-da8dfbb2c7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:40.280826 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b13fa-de3c-409f-a5d8-da8dfbb2c7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:40.312049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b90b13fa-de3c-409f-a5d8-da8dfbb2c7f3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:40.314306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b90b13fa-de3c-409f-a5d8-da8dfbb2c7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:40.333786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f30ac-fb5c-43c9-affa-220797e4a67f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:40.375863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b6763-5e41-4ae8-8739-b69c3de06b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:40.376745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b6763-5e41-4ae8-8739-b69c3de06b2d None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:40.377030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b6763-5e41-4ae8-8739-b69c3de06b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:40.377307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b6763-5e41-4ae8-8739-b69c3de06b2d None admin] PATH_INFO: `/v3/domains/default/users/3b98d845867d491c91aeca1e12cf3ace/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:40.379642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b6763-5e41-4ae8-8739-b69c3de06b2d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3b98d845867d491c91aeca1e12cf3ace, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:40.381053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b6763-5e41-4ae8-8739-b69c3de06b2d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:40.400209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 20/42] 10.222.0.46 () {58 vars in 1361 bytes} [Thu May 26 18:02:40 2022] PUT /identity/v3/domains/default/users/3b98d845867d491c91aeca1e12cf3ace/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:40.402643 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b6763-5e41-4ae8-8739-b69c3de06b2d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:40.403026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 23/43] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:40.428524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b13fa-de3c-409f-a5d8-da8dfbb2c7f3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:40.430636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:40.471254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:40.534327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b6763-5e41-4ae8-8739-b69c3de06b2d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:40.558907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b13fa-de3c-409f-a5d8-da8dfbb2c7f3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:40.572731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86c63d-8fd1-4944-a3b8-34ee41da3b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:40.573949 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86c63d-8fd1-4944-a3b8-34ee41da3b78 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:40.574819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86c63d-8fd1-4944-a3b8-34ee41da3b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:40.575322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86c63d-8fd1-4944-a3b8-34ee41da3b78 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:40.576273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86c63d-8fd1-4944-a3b8-34ee41da3b78 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:40.578468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86c63d-8fd1-4944-a3b8-34ee41da3b78 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:40.599498 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 21/44] 10.222.0.46 () {60 vars in 1131 bytes} [Thu May 26 18:02:40 2022] POST /identity/v3/roles => generated 223 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:02:40.605754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d052a-7273-4b9e-abf4-da3d6fe5651e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:40.605754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d052a-7273-4b9e-abf4-da3d6fe5651e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:40.605754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d052a-7273-4b9e-abf4-da3d6fe5651e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:40.605754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d052a-7273-4b9e-abf4-da3d6fe5651e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:40.606313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d052a-7273-4b9e-abf4-da3d6fe5651e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:40.606313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d052a-7273-4b9e-abf4-da3d6fe5651e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:40.608344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 24/45] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:02:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:02:40.616031 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86c63d-8fd1-4944-a3b8-34ee41da3b78 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:40.695654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:40.734412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86c63d-8fd1-4944-a3b8-34ee41da3b78 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:40.772273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17369dc0-0c0c-4ecc-9d2c-19ac992222c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:40.772967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17369dc0-0c0c-4ecc-9d2c-19ac992222c6 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:40.773249 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17369dc0-0c0c-4ecc-9d2c-19ac992222c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:40.773549 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17369dc0-0c0c-4ecc-9d2c-19ac992222c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:40.774313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17369dc0-0c0c-4ecc-9d2c-19ac992222c6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:40.775773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17369dc0-0c0c-4ecc-9d2c-19ac992222c6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:40.806445 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 22/46] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:02:40 2022] POST /identity/v3/projects => generated 312 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:02:42.692392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3b5f0-80a5-4db6-bdaf-8598691ec8af None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:42.692926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3b5f0-80a5-4db6-bdaf-8598691ec8af None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:42.693009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3b5f0-80a5-4db6-bdaf-8598691ec8af None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:42.694352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 25/47] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:42.699673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb1b2f-0362-46f8-8d5d-addc6226caa5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:42.700056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb1b2f-0362-46f8-8d5d-addc6226caa5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:42.700237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb1b2f-0362-46f8-8d5d-addc6226caa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:42.718377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1fcb1b2f-0362-46f8-8d5d-addc6226caa5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:42.719515 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1fcb1b2f-0362-46f8-8d5d-addc6226caa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:42.787144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 23/48] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:42.810136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3b5f0-80a5-4db6-bdaf-8598691ec8af None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:42.917401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3b5f0-80a5-4db6-bdaf-8598691ec8af None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:42.919034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:42.955538 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93392a9e-f19d-4f13-b57d-7baa5beb59d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:42.956251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93392a9e-f19d-4f13-b57d-7baa5beb59d5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:42.956533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93392a9e-f19d-4f13-b57d-7baa5beb59d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:42.956805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93392a9e-f19d-4f13-b57d-7baa5beb59d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:42.969196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93392a9e-f19d-4f13-b57d-7baa5beb59d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-93392a9e-f19d-4f13-b57d-7baa5beb59d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:42.977058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:42.978057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:02:42.978845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 26/49] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:02:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:42.985827 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb1b2f-0362-46f8-8d5d-addc6226caa5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:43.017337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279873c-b060-4c62-b8c6-37963ccd7a51 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:43.017631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279873c-b060-4c62-b8c6-37963ccd7a51 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:43.017944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279873c-b060-4c62-b8c6-37963ccd7a51 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:43.019301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 27/50] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:02:43.024089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6173d-9c65-4904-9716-021c556349da None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:43.024389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6173d-9c65-4904-9716-021c556349da None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:43.024672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6173d-9c65-4904-9716-021c556349da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:43.036445 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:43.042488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-39a6173d-9c65-4904-9716-021c556349da None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:43.043682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-39a6173d-9c65-4904-9716-021c556349da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:43.088728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb1b2f-0362-46f8-8d5d-addc6226caa5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:43.112713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 28/51] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:02:43.123444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1340b-a6a1-4d28-8794-85c57964aaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:43.124121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1340b-a6a1-4d28-8794-85c57964aaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:43.124406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1340b-a6a1-4d28-8794-85c57964aaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:43.124694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1340b-a6a1-4d28-8794-85c57964aaa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:43.125443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1340b-a6a1-4d28-8794-85c57964aaa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:43.125781 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1340b-a6a1-4d28-8794-85c57964aaa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:43.127184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1340b-a6a1-4d28-8794-85c57964aaa4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:43.135421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 24/52] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:02:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:43.136539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6173d-9c65-4904-9716-021c556349da None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:43.141083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1340b-a6a1-4d28-8794-85c57964aaa4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:43.228653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:43.238736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6173d-9c65-4904-9716-021c556349da None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:43.242343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1340b-a6a1-4d28-8794-85c57964aaa4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:43.255408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:43.277665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881eea2-503e-4fda-abb2-737b1ff30186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:43.277889 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c5337-7c22-4ec2-b7fa-af5c3679cfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:43.278536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c5337-7c22-4ec2-b7fa-af5c3679cfa2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:43.278633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881eea2-503e-4fda-abb2-737b1ff30186 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:43.278820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c5337-7c22-4ec2-b7fa-af5c3679cfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:43.278919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881eea2-503e-4fda-abb2-737b1ff30186 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:43.279092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c5337-7c22-4ec2-b7fa-af5c3679cfa2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:43.279175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881eea2-503e-4fda-abb2-737b1ff30186 None admin] PATH_INFO: `/v3/users/3b98d845867d491c91aeca1e12cf3ace` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:43.280854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881eea2-503e-4fda-abb2-737b1ff30186 None admin] RBAC: Authorizing `identity:get_user(user_id=3b98d845867d491c91aeca1e12cf3ace)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:43.281157 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c5337-7c22-4ec2-b7fa-af5c3679cfa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:43.282388 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881eea2-503e-4fda-abb2-737b1ff30186 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:43.282725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c5337-7c22-4ec2-b7fa-af5c3679cfa2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:43.284042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 25/53] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:43 2022] GET /identity/v3/users/3b98d845867d491c91aeca1e12cf3ace => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:02:43.284764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 29/54] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:02:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:02:43.289581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881eea2-503e-4fda-abb2-737b1ff30186 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:43.290811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c5337-7c22-4ec2-b7fa-af5c3679cfa2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:43.392288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c5337-7c22-4ec2-b7fa-af5c3679cfa2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:43.392635 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881eea2-503e-4fda-abb2-737b1ff30186 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:43.429346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f43a7-a577-46a5-915b-7d6453234799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:43.429346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f43a7-a577-46a5-915b-7d6453234799 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:43.430650 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f43a7-a577-46a5-915b-7d6453234799 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:43.430650 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f43a7-a577-46a5-915b-7d6453234799 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:43.430812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f43a7-a577-46a5-915b-7d6453234799 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:43.431642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a5257-5898-4920-9d9f-147a203729e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:43.432237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f43a7-a577-46a5-915b-7d6453234799 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:43.432433 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a5257-5898-4920-9d9f-147a203729e4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:43.432720 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a5257-5898-4920-9d9f-147a203729e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:43.432996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a5257-5898-4920-9d9f-147a203729e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:43.434219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a5257-5898-4920-9d9f-147a203729e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:43.435665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a5257-5898-4920-9d9f-147a203729e4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:43.437286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 26/55] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:02:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:02:43.443139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a5257-5898-4920-9d9f-147a203729e4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:43.459719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 30/56] 10.222.0.46 () {60 vars in 1140 bytes} [Thu May 26 18:02:43 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:02:43.500276 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:43.559492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a5257-5898-4920-9d9f-147a203729e4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:43.596251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04741053-61a1-4afa-9558-bfb77d24cbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:43.596929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04741053-61a1-4afa-9558-bfb77d24cbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:43.597207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04741053-61a1-4afa-9558-bfb77d24cbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:43.597478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04741053-61a1-4afa-9558-bfb77d24cbf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:43.598353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04741053-61a1-4afa-9558-bfb77d24cbf2 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.domain.id=default, user.id=3b98d845867d491c91aeca1e12cf3ace) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:43.598664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04741053-61a1-4afa-9558-bfb77d24cbf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:43.600129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04741053-61a1-4afa-9558-bfb77d24cbf2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:43.610165 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 27/57] 10.222.0.46 () {56 vars in 1322 bytes} [Thu May 26 18:02:43 2022] GET /identity/v3/role_assignments?user.id=3b98d845867d491c91aeca1e12cf3ace&scope.domain.id=default&role.id=584beecb022b422f820e77f7623ada5e => generated 530 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:02:45.698275 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a08fd-0b4e-44c1-961c-c57707a056d3 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:45.698786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a08fd-0b4e-44c1-961c-c57707a056d3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:45.698867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a08fd-0b4e-44c1-961c-c57707a056d3 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:45.700200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 31/58] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:45.706169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e755a-09c0-4d99-81fe-3fe6c16e5e17 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:45.706567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e755a-09c0-4d99-81fe-3fe6c16e5e17 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:45.706907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e755a-09c0-4d99-81fe-3fe6c16e5e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:45.726231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2b2e755a-09c0-4d99-81fe-3fe6c16e5e17 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:45.727433 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2b2e755a-09c0-4d99-81fe-3fe6c16e5e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:45.792790 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 28/59] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:45.818269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a08fd-0b4e-44c1-961c-c57707a056d3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:45.833582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:45.922016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a08fd-0b4e-44c1-961c-c57707a056d3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:45.954602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a9330-5d8b-4d8c-ba4c-36f5ed63ccdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:45.955318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a9330-5d8b-4d8c-ba4c-36f5ed63ccdc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:45.955599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a9330-5d8b-4d8c-ba4c-36f5ed63ccdc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:45.955871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a9330-5d8b-4d8c-ba4c-36f5ed63ccdc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:45.957024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a9330-5d8b-4d8c-ba4c-36f5ed63ccdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:45.958512 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a9330-5d8b-4d8c-ba4c-36f5ed63ccdc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:45.960099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 32/60] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:02:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:45.965749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e755a-09c0-4d99-81fe-3fe6c16e5e17 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:46.008381 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:46.063297 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e755a-09c0-4d99-81fe-3fe6c16e5e17 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:46.096046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2dd78-1c6c-4732-b58e-765d460bb77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:46.096706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2dd78-1c6c-4732-b58e-765d460bb77c None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:46.096984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2dd78-1c6c-4732-b58e-765d460bb77c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:46.097255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2dd78-1c6c-4732-b58e-765d460bb77c None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:46.097973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2dd78-1c6c-4732-b58e-765d460bb77c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:46.099396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2dd78-1c6c-4732-b58e-765d460bb77c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:46.125589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 29/61] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:02:45 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:02:49.553605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b289a80-7b5f-4f0c-95aa-753667788f8d None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:49.556325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b289a80-7b5f-4f0c-95aa-753667788f8d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:49.556325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b289a80-7b5f-4f0c-95aa-753667788f8d None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:49.556325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 33/62] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:49.561524 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe6742-6c0e-49e6-8cbe-67994cbfc21c None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:49.562367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe6742-6c0e-49e6-8cbe-67994cbfc21c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:49.562844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe6742-6c0e-49e6-8cbe-67994cbfc21c None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:49.564403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 30/63] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:49.565912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc72ba2-b2a5-4886-919b-46c4f2cd0398 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:49.566429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc72ba2-b2a5-4886-919b-46c4f2cd0398 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:49.566896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc72ba2-b2a5-4886-919b-46c4f2cd0398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:49.582325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021659e-334f-4afc-8cd3-7ba343b51ca2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:49.582919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021659e-334f-4afc-8cd3-7ba343b51ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:49.583393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021659e-334f-4afc-8cd3-7ba343b51ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:49.601720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-fcc72ba2-b2a5-4886-919b-46c4f2cd0398 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:49.603436 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-fcc72ba2-b2a5-4886-919b-46c4f2cd0398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:49.611663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b021659e-334f-4afc-8cd3-7ba343b51ca2 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:49.613129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b021659e-334f-4afc-8cd3-7ba343b51ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:49.735171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 31/64] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:49.752071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 34/65] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:49.759307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021659e-334f-4afc-8cd3-7ba343b51ca2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:49.775783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc72ba2-b2a5-4886-919b-46c4f2cd0398 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:49.797908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:49.800486 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:49.947489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021659e-334f-4afc-8cd3-7ba343b51ca2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:49.960139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc72ba2-b2a5-4886-919b-46c4f2cd0398 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:49.988413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde4932-f32f-4e22-8dcf-bda1429adb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:49.989223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde4932-f32f-4e22-8dcf-bda1429adb0b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:49.989507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde4932-f32f-4e22-8dcf-bda1429adb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:49.989817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde4932-f32f-4e22-8dcf-bda1429adb0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.003351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fd4bc-46e2-4e88-b942-40eb70afc012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.004529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fd4bc-46e2-4e88-b942-40eb70afc012 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.004634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde4932-f32f-4e22-8dcf-bda1429adb0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:50.005103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fd4bc-46e2-4e88-b942-40eb70afc012 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.005599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fd4bc-46e2-4e88-b942-40eb70afc012 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-bdde4932-f32f-4e22-8dcf-bda1429adb0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:50.014504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:50.015423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:02:50.016177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 32/66] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:02:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:50.017577 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1842b8-a0e3-4560-933d-2b39ea35e6ee None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.017577 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1842b8-a0e3-4560-933d-2b39ea35e6ee None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.017744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1842b8-a0e3-4560-933d-2b39ea35e6ee None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.021252 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fd4bc-46e2-4e88-b942-40eb70afc012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:50.024035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 33/67] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:50.024708 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1842b8-a0e3-4560-933d-2b39ea35e6ee None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-721fd4bc-46e2-4e88-b942-40eb70afc012 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:50.030424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:50.031316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:50.032058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:50.032058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:50.032058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:50.032058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:50.032058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:50.032058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:50.032058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:50.032058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:50.032058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:50.032058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:50.032058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:02:50.032058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:02:50.034759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 35/68] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:02:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:02:50.036438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ba9fd-1e70-45de-8725-50acff624bbc None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.036927 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ba9fd-1e70-45de-8725-50acff624bbc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.037453 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ba9fd-1e70-45de-8725-50acff624bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.056159 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2b6ba9fd-1e70-45de-8725-50acff624bbc None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:50.057811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2b6ba9fd-1e70-45de-8725-50acff624bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:50.144708 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:50.146485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:50.153388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 36/69] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:50.155388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ba9fd-1e70-45de-8725-50acff624bbc None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.164945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1842b8-a0e3-4560-933d-2b39ea35e6ee None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.210173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3ff07-740b-44c8-860b-65bbdfb0590b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.211287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3ff07-740b-44c8-860b-65bbdfb0590b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.211780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3ff07-740b-44c8-860b-65bbdfb0590b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.212236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3ff07-740b-44c8-860b-65bbdfb0590b None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.213647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3ff07-740b-44c8-860b-65bbdfb0590b None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:50.214400 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3ff07-740b-44c8-860b-65bbdfb0590b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:50.217238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3ff07-740b-44c8-860b-65bbdfb0590b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:50.234568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 34/70] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:50.236663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341f72b-e5b6-4f85-a6b4-e2d47f9ce93b None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.237164 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341f72b-e5b6-4f85-a6b4-e2d47f9ce93b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.237629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341f72b-e5b6-4f85-a6b4-e2d47f9ce93b None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.241975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 35/71] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:50.243608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341f72b-e5b6-4f85-a6b4-e2d47f9ce93b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.279064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ba9fd-1e70-45de-8725-50acff624bbc None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.319351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f262bff-4ab2-465e-984f-bcbfdc320c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.319928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f262bff-4ab2-465e-984f-bcbfdc320c32 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.320212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f262bff-4ab2-465e-984f-bcbfdc320c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.320484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f262bff-4ab2-465e-984f-bcbfdc320c32 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.321238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f262bff-4ab2-465e-984f-bcbfdc320c32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:50.321542 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f262bff-4ab2-465e-984f-bcbfdc320c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:50.322996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f262bff-4ab2-465e-984f-bcbfdc320c32 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:50.331821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 37/72] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:50.331821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f262bff-4ab2-465e-984f-bcbfdc320c32 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.374233 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341f72b-e5b6-4f85-a6b4-e2d47f9ce93b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.399471 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:50.408806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1b3c2-9759-474b-910b-4459a316ffc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.409909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1b3c2-9759-474b-910b-4459a316ffc1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.410193 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1b3c2-9759-474b-910b-4459a316ffc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.410471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1b3c2-9759-474b-910b-4459a316ffc1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.421365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1b3c2-9759-474b-910b-4459a316ffc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-17c1b3c2-9759-474b-910b-4459a316ffc1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:50.429490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:50.438147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 36/73] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:50.438894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fd7ac-9c5b-4216-b9b9-e99898623ef4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.439552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fd7ac-9c5b-4216-b9b9-e99898623ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.439552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fd7ac-9c5b-4216-b9b9-e99898623ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.442540 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f262bff-4ab2-465e-984f-bcbfdc320c32 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.450292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e23fd7ac-9c5b-4216-b9b9-e99898623ef4 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:50.451776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e23fd7ac-9c5b-4216-b9b9-e99898623ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:50.455423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:50.477129 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c57770-493b-465e-aceb-0108f87058de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.477941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c57770-493b-465e-aceb-0108f87058de None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.478222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c57770-493b-465e-aceb-0108f87058de None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.478504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c57770-493b-465e-aceb-0108f87058de None admin] PATH_INFO: `/v3/users/331362e99eb14800b4d52472fd41a629` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.492611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c57770-493b-465e-aceb-0108f87058de None admin] RBAC: Authorizing `identity:get_user(user_id=331362e99eb14800b4d52472fd41a629)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:50.494123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c57770-493b-465e-aceb-0108f87058de None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:50.495819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 38/74] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/users/331362e99eb14800b4d52472fd41a629 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:50.497060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cdd18-bf10-4ae6-9482-56c8a4c2b045 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.497342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cdd18-bf10-4ae6-9482-56c8a4c2b045 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.497613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cdd18-bf10-4ae6-9482-56c8a4c2b045 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 39/75] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:50.500138 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cdd18-bf10-4ae6-9482-56c8a4c2b045 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.527495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 37/76] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:50.528369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fd7ac-9c5b-4216-b9b9-e99898623ef4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.619020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cdd18-bf10-4ae6-9482-56c8a4c2b045 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.627956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fd7ac-9c5b-4216-b9b9-e99898623ef4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.639956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:50.654291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824a938-9a79-4e2f-b239-146516000cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.655845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824a938-9a79-4e2f-b239-146516000cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.656121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824a938-9a79-4e2f-b239-146516000cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.656484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824a938-9a79-4e2f-b239-146516000cf9 None admin] PATH_INFO: `/v3/users/3b98d845867d491c91aeca1e12cf3ace` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.657681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824a938-9a79-4e2f-b239-146516000cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=3b98d845867d491c91aeca1e12cf3ace)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:50.659164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824a938-9a79-4e2f-b239-146516000cf9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:50.661018 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 40/77] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/users/3b98d845867d491c91aeca1e12cf3ace => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:50.661804 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824a938-9a79-4e2f-b239-146516000cf9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.667501 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496d5f4-73f0-43b3-afe5-b334c92ce6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.668341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496d5f4-73f0-43b3-afe5-b334c92ce6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.669457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496d5f4-73f0-43b3-afe5-b334c92ce6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.669751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496d5f4-73f0-43b3-afe5-b334c92ce6a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496d5f4-73f0-43b3-afe5-b334c92ce6a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:50.671125 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496d5f4-73f0-43b3-afe5-b334c92ce6a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:50.672524 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496d5f4-73f0-43b3-afe5-b334c92ce6a7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:50.680974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 38/78] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:50.682413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbee873-7b1b-4011-9c2a-38122ce55399 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.682704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbee873-7b1b-4011-9c2a-38122ce55399 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.682980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbee873-7b1b-4011-9c2a-38122ce55399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.687991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:50.702671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-cdbee873-7b1b-4011-9c2a-38122ce55399 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:50.707564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cdbee873-7b1b-4011-9c2a-38122ce55399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:50.778302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824a938-9a79-4e2f-b239-146516000cf9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.790199 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 39/79] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:50.791050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbee873-7b1b-4011-9c2a-38122ce55399 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.816913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff59177-c130-429d-90cc-899b8de006e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.817567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff59177-c130-429d-90cc-899b8de006e6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.817883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff59177-c130-429d-90cc-899b8de006e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.818164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff59177-c130-429d-90cc-899b8de006e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.873436 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:50.886069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff59177-c130-429d-90cc-899b8de006e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:50.887874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff59177-c130-429d-90cc-899b8de006e6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:50.891327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbee873-7b1b-4011-9c2a-38122ce55399 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.920762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:50.926408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf7dcc-c60e-4a5c-935a-e76fdcabbdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.927212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf7dcc-c60e-4a5c-935a-e76fdcabbdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.927495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf7dcc-c60e-4a5c-935a-e76fdcabbdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.927776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf7dcc-c60e-4a5c-935a-e76fdcabbdc8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.939002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf7dcc-c60e-4a5c-935a-e76fdcabbdc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-bebf7dcc-c60e-4a5c-935a-e76fdcabbdc8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:50.946999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:50.948108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 40/80] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:50.949075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf7dcc-c60e-4a5c-935a-e76fdcabbdc8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.992161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 41/81] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:02:50.993026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff59177-c130-429d-90cc-899b8de006e6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.063811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf7dcc-c60e-4a5c-935a-e76fdcabbdc8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.095700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:51.099840 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30ee89-7797-4b63-af1c-68dd16805e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.100501 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30ee89-7797-4b63-af1c-68dd16805e78 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.100772 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30ee89-7797-4b63-af1c-68dd16805e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.101048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30ee89-7797-4b63-af1c-68dd16805e78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.108989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff59177-c130-429d-90cc-899b8de006e6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.146572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95c193-4f7b-46f5-91ef-73f861966b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.147461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95c193-4f7b-46f5-91ef-73f861966b64 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.147746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95c193-4f7b-46f5-91ef-73f861966b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.148023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95c193-4f7b-46f5-91ef-73f861966b64 None admin] PATH_INFO: `/v3/users/331362e99eb14800b4d52472fd41a629` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.149286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95c193-4f7b-46f5-91ef-73f861966b64 None admin] RBAC: Authorizing `identity:get_user(user_id=331362e99eb14800b4d52472fd41a629)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.150819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95c193-4f7b-46f5-91ef-73f861966b64 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.152533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 42/82] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/users/331362e99eb14800b4d52472fd41a629 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:51.155628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95c193-4f7b-46f5-91ef-73f861966b64 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.176225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:51.188494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30ee89-7797-4b63-af1c-68dd16805e78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.190147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30ee89-7797-4b63-af1c-68dd16805e78 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.262823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95c193-4f7b-46f5-91ef-73f861966b64 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.303286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 41/83] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:02:51.304133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30ee89-7797-4b63-af1c-68dd16805e78 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.304418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdc1c5-ddf3-46fd-a5d0-6032923fdf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.305219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdc1c5-ddf3-46fd-a5d0-6032923fdf36 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.305501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdc1c5-ddf3-46fd-a5d0-6032923fdf36 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.305803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdc1c5-ddf3-46fd-a5d0-6032923fdf36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.307042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdc1c5-ddf3-46fd-a5d0-6032923fdf36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.308534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdc1c5-ddf3-46fd-a5d0-6032923fdf36 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.310189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 43/84] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:02:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:51.311495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdc1c5-ddf3-46fd-a5d0-6032923fdf36 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.339405 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:51.405620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:51.415375 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30ee89-7797-4b63-af1c-68dd16805e78 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.418440 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdc1c5-ddf3-46fd-a5d0-6032923fdf36 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.449896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cc4a0-6371-423e-a6fb-111e470db215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.450562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cc4a0-6371-423e-a6fb-111e470db215 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.450848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cc4a0-6371-423e-a6fb-111e470db215 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.451171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cc4a0-6371-423e-a6fb-111e470db215 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.452065 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cc4a0-6371-423e-a6fb-111e470db215 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:51.452377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cc4a0-6371-423e-a6fb-111e470db215 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.453377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e810b-0219-402a-8bcb-64e4c47c8226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.453794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cc4a0-6371-423e-a6fb-111e470db215 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.454266 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e810b-0219-402a-8bcb-64e4c47c8226 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.454548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e810b-0219-402a-8bcb-64e4c47c8226 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.454824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e810b-0219-402a-8bcb-64e4c47c8226 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.456000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e810b-0219-402a-8bcb-64e4c47c8226 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.457463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e810b-0219-402a-8bcb-64e4c47c8226 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.459200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 44/85] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:02:51 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:51.459958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e810b-0219-402a-8bcb-64e4c47c8226 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.462283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 42/86] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:02:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:51.462908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cc4a0-6371-423e-a6fb-111e470db215 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.556743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:51.568851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cc4a0-6371-423e-a6fb-111e470db215 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.571143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e810b-0219-402a-8bcb-64e4c47c8226 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.602860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7a766-126b-46b3-a8ac-0fab1d1fc327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.603731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7a766-126b-46b3-a8ac-0fab1d1fc327 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.604014 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7a766-126b-46b3-a8ac-0fab1d1fc327 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.604291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7a766-126b-46b3-a8ac-0fab1d1fc327 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.605474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7a766-126b-46b3-a8ac-0fab1d1fc327 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.606953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7a766-126b-46b3-a8ac-0fab1d1fc327 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.608698 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 43/87] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:02:51 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:51.609426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7a766-126b-46b3-a8ac-0fab1d1fc327 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.617660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452566f-2055-4b7b-a79d-c2f5904c01ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.618342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452566f-2055-4b7b-a79d-c2f5904c01ca None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.618628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452566f-2055-4b7b-a79d-c2f5904c01ca None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.619500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452566f-2055-4b7b-a79d-c2f5904c01ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.634627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:51.695979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452566f-2055-4b7b-a79d-c2f5904c01ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.697511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452566f-2055-4b7b-a79d-c2f5904c01ca None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.729323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7a766-126b-46b3-a8ac-0fab1d1fc327 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.770925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685da9d-2091-48a5-a07b-983e3ec971a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.771886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685da9d-2091-48a5-a07b-983e3ec971a3 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.772390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685da9d-2091-48a5-a07b-983e3ec971a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.772850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685da9d-2091-48a5-a07b-983e3ec971a3 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.774078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685da9d-2091-48a5-a07b-983e3ec971a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.775883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685da9d-2091-48a5-a07b-983e3ec971a3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.798316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 44/88] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:02:51 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:02:51.799416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685da9d-2091-48a5-a07b-983e3ec971a3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.822479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 45/89] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:02:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:02:51.823908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452566f-2055-4b7b-a79d-c2f5904c01ca None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.825106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:51.910835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:51.938779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685da9d-2091-48a5-a07b-983e3ec971a3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.985448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19beb02-9bd3-4270-bc2f-2d804c45c626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.986374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19beb02-9bd3-4270-bc2f-2d804c45c626 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.986664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19beb02-9bd3-4270-bc2f-2d804c45c626 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.986944 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19beb02-9bd3-4270-bc2f-2d804c45c626 None admin] PATH_INFO: `/v3/users/331362e99eb14800b4d52472fd41a629` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.987036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452566f-2055-4b7b-a79d-c2f5904c01ca None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.988355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19beb02-9bd3-4270-bc2f-2d804c45c626 None admin] RBAC: Authorizing `identity:get_user(user_id=331362e99eb14800b4d52472fd41a629)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.989847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19beb02-9bd3-4270-bc2f-2d804c45c626 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.991502 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 45/90] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:51 2022] GET /identity/v3/users/331362e99eb14800b4d52472fd41a629 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:51.992285 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19beb02-9bd3-4270-bc2f-2d804c45c626 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:52.036335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5d366-62b2-44fb-9f94-977542d1ce6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:52.037761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d366-62b2-44fb-9f94-977542d1ce6a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.037761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d366-62b2-44fb-9f94-977542d1ce6a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.037944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d366-62b2-44fb-9f94-977542d1ce6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.039939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5d366-62b2-44fb-9f94-977542d1ce6a None admin] RBAC: Adding query filter params (role.id=e1a5192e7abc4046bc2c0673176ea1b6, scope.project.id=f5b2e96e553d496d96e8f916fb522477, user.id=331362e99eb14800b4d52472fd41a629) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:52.039939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5d366-62b2-44fb-9f94-977542d1ce6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:52.041290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5d366-62b2-44fb-9f94-977542d1ce6a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:52.050721 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 46/91] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:02:51 2022] GET /identity/v3/role_assignments?user.id=331362e99eb14800b4d52472fd41a629&scope.project.id=f5b2e96e553d496d96e8f916fb522477&role.id=e1a5192e7abc4046bc2c0673176ea1b6 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:52.051610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5d366-62b2-44fb-9f94-977542d1ce6a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:52.138434 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:52.160822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19beb02-9bd3-4270-bc2f-2d804c45c626 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:52.203473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5d366-62b2-44fb-9f94-977542d1ce6a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:52.228419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048ef76-f621-462b-ae3d-52bc01a0a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:52.229962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048ef76-f621-462b-ae3d-52bc01a0a5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.230403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048ef76-f621-462b-ae3d-52bc01a0a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.230832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048ef76-f621-462b-ae3d-52bc01a0a5f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.233202 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048ef76-f621-462b-ae3d-52bc01a0a5f9 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=f5b2e96e553d496d96e8f916fb522477, user.id=3b98d845867d491c91aeca1e12cf3ace) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:52.233776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048ef76-f621-462b-ae3d-52bc01a0a5f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:52.237625 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048ef76-f621-462b-ae3d-52bc01a0a5f9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:52.248644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 46/92] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:02:51 2022] GET /identity/v3/role_assignments?user.id=3b98d845867d491c91aeca1e12cf3ace&scope.project.id=f5b2e96e553d496d96e8f916fb522477&role.id=584beecb022b422f820e77f7623ada5e => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:52.249665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048ef76-f621-462b-ae3d-52bc01a0a5f9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:52.251705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e65bd8-522b-4517-93b1-79efb10ce6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:52.252750 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e65bd8-522b-4517-93b1-79efb10ce6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.253082 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e65bd8-522b-4517-93b1-79efb10ce6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.253428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e65bd8-522b-4517-93b1-79efb10ce6d5 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.255001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e65bd8-522b-4517-93b1-79efb10ce6d5 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:52.256686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e65bd8-522b-4517-93b1-79efb10ce6d5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:52.258972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 47/93] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:02:52 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:52.266909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e65bd8-522b-4517-93b1-79efb10ce6d5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:52.268751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:52.387758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048ef76-f621-462b-ae3d-52bc01a0a5f9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:52.427087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d1170-ca56-46dc-a484-e994a7d93684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:52.427758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d1170-ca56-46dc-a484-e994a7d93684 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.428043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d1170-ca56-46dc-a484-e994a7d93684 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.428320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d1170-ca56-46dc-a484-e994a7d93684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.488241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e65bd8-522b-4517-93b1-79efb10ce6d5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:52.504825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d1170-ca56-46dc-a484-e994a7d93684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:52.506387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d1170-ca56-46dc-a484-e994a7d93684 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:52.518937 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:52.540982 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:52.559457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e507eb-77e9-40a9-bc52-c49e165d4bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:52.560141 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e507eb-77e9-40a9-bc52-c49e165d4bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.560474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e507eb-77e9-40a9-bc52-c49e165d4bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.560941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e507eb-77e9-40a9-bc52-c49e165d4bb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.561941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e507eb-77e9-40a9-bc52-c49e165d4bb5 None admin] RBAC: Adding query filter params (role.id=506b3cf33324478abe0f72a9d408aa54, scope.project.id=f5b2e96e553d496d96e8f916fb522477, user.id=331362e99eb14800b4d52472fd41a629) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:52.562266 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e507eb-77e9-40a9-bc52-c49e165d4bb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:52.563819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e507eb-77e9-40a9-bc52-c49e165d4bb5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:52.580735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 48/94] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:02:52 2022] GET /identity/v3/role_assignments?user.id=331362e99eb14800b4d52472fd41a629&scope.project.id=f5b2e96e553d496d96e8f916fb522477&role.id=506b3cf33324478abe0f72a9d408aa54 => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:52.626043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 47/95] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:02:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:02:52.631261 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e507eb-77e9-40a9-bc52-c49e165d4bb5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:52.805561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e507eb-77e9-40a9-bc52-c49e165d4bb5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:52.859061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a1f80-570a-4ad7-9623-0520da1b17eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:52.860339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a1f80-570a-4ad7-9623-0520da1b17eb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.860823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a1f80-570a-4ad7-9623-0520da1b17eb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.861286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a1f80-570a-4ad7-9623-0520da1b17eb None admin] PATH_INFO: `/v3/projects/db9fa468925d45d9901c0bed2034e05f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.862814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a1f80-570a-4ad7-9623-0520da1b17eb None admin] RBAC: Authorizing `identity:get_project(project_id=db9fa468925d45d9901c0bed2034e05f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:52.864546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a1f80-570a-4ad7-9623-0520da1b17eb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:52.866559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 49/96] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:02:52 2022] GET /identity/v3/projects/db9fa468925d45d9901c0bed2034e05f => generated 312 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:52.872713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d1170-ca56-46dc-a484-e994a7d93684 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:52.994727 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d1170-ca56-46dc-a484-e994a7d93684 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:53.006871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:53.032523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520638f-1bd6-4658-b813-73447bfab9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:53.033188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520638f-1bd6-4658-b813-73447bfab9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:53.033474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520638f-1bd6-4658-b813-73447bfab9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:53.033818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520638f-1bd6-4658-b813-73447bfab9b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:53.035090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520638f-1bd6-4658-b813-73447bfab9b6 None admin] RBAC: Adding query filter params (role.id=e1a5192e7abc4046bc2c0673176ea1b6, scope.project.id=db9fa468925d45d9901c0bed2034e05f, user.id=331362e99eb14800b4d52472fd41a629) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:53.035405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520638f-1bd6-4658-b813-73447bfab9b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:53.036884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520638f-1bd6-4658-b813-73447bfab9b6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:53.043394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 48/97] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:02:52 2022] GET /identity/v3/role_assignments?user.id=331362e99eb14800b4d52472fd41a629&scope.project.id=db9fa468925d45d9901c0bed2034e05f&role.id=e1a5192e7abc4046bc2c0673176ea1b6 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:55.195198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0b040-f6c6-4c82-a67c-948da9a91b76 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:55.196261 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0b040-f6c6-4c82-a67c-948da9a91b76 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:55.196748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0b040-f6c6-4c82-a67c-948da9a91b76 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:55.198720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 50/98] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:55.204255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf129b14-4bed-465f-b070-a2bc7597d6db None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:55.204733 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf129b14-4bed-465f-b070-a2bc7597d6db None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:55.204816 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf129b14-4bed-465f-b070-a2bc7597d6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:55.228741 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bf129b14-4bed-465f-b070-a2bc7597d6db None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:55.229977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bf129b14-4bed-465f-b070-a2bc7597d6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:55.320389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 49/99] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:55.353997 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0b040-f6c6-4c82-a67c-948da9a91b76 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:55.441351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:55.495189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0b040-f6c6-4c82-a67c-948da9a91b76 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:55.546332 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055855e-ebf7-4fba-95e6-c2504217c51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:55.547149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055855e-ebf7-4fba-95e6-c2504217c51a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:55.547443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055855e-ebf7-4fba-95e6-c2504217c51a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:55.547726 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055855e-ebf7-4fba-95e6-c2504217c51a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:55.562863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055855e-ebf7-4fba-95e6-c2504217c51a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-7055855e-ebf7-4fba-95e6-c2504217c51a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:55.571397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:55.572375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:02:55.573153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 51/100] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:02:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:55.580377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf129b14-4bed-465f-b070-a2bc7597d6db None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:55.608232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afb415-7a4e-4e9f-a42d-b509cf5b408a None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:55.608703 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afb415-7a4e-4e9f-a42d-b509cf5b408a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:55.608880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afb415-7a4e-4e9f-a42d-b509cf5b408a None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:55.610413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 52/101] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:55.617337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83443eec-8128-4629-8a72-7543d34777b2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:55.617863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83443eec-8128-4629-8a72-7543d34777b2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:55.618305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83443eec-8128-4629-8a72-7543d34777b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:55.647265 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-83443eec-8128-4629-8a72-7543d34777b2 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:55.655913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-83443eec-8128-4629-8a72-7543d34777b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:55.775288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf129b14-4bed-465f-b070-a2bc7597d6db None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:55.778869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:55.819904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043fd30-095e-4a41-9ca6-67550faf12fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:55.820911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043fd30-095e-4a41-9ca6-67550faf12fa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:55.821403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043fd30-095e-4a41-9ca6-67550faf12fa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:55.821890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043fd30-095e-4a41-9ca6-67550faf12fa None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:55.823053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043fd30-095e-4a41-9ca6-67550faf12fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:55.823525 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043fd30-095e-4a41-9ca6-67550faf12fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:55.825096 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043fd30-095e-4a41-9ca6-67550faf12fa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:55.835881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 50/102] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:02:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:55.837482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccdb89-6976-48eb-94f9-e7e308adf690 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:55.837990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccdb89-6976-48eb-94f9-e7e308adf690 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:55.838803 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccdb89-6976-48eb-94f9-e7e308adf690 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:55.840226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 51/103] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:55.842318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccdb89-6976-48eb-94f9-e7e308adf690 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:55.851385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 53/104] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:02:55.852733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb50cf-f1b3-41dd-89dc-9dce4e829f76 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:55.853027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb50cf-f1b3-41dd-89dc-9dce4e829f76 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:55.853378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb50cf-f1b3-41dd-89dc-9dce4e829f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:55.902801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1bcb50cf-f1b3-41dd-89dc-9dce4e829f76 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:55.903588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1bcb50cf-f1b3-41dd-89dc-9dce4e829f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:55.953133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccdb89-6976-48eb-94f9-e7e308adf690 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:55.990254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3c646-c467-49f1-ad1d-9fba5c8dde42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:55.991495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3c646-c467-49f1-ad1d-9fba5c8dde42 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:55.991963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3c646-c467-49f1-ad1d-9fba5c8dde42 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:55.992451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3c646-c467-49f1-ad1d-9fba5c8dde42 None admin] PATH_INFO: `/v3/users/331362e99eb14800b4d52472fd41a629` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:55.993907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3c646-c467-49f1-ad1d-9fba5c8dde42 None admin] RBAC: Authorizing `identity:get_user(user_id=331362e99eb14800b4d52472fd41a629)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:55.995632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3c646-c467-49f1-ad1d-9fba5c8dde42 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:55.996163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:55.997778 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 52/105] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:55 2022] GET /identity/v3/users/331362e99eb14800b4d52472fd41a629 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:02:55.998908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3c646-c467-49f1-ad1d-9fba5c8dde42 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:56.030863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 54/106] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:56.039295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:56.039523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3bed9-b1aa-4079-9e8c-5e008770a039 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.040239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3bed9-b1aa-4079-9e8c-5e008770a039 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.040523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3bed9-b1aa-4079-9e8c-5e008770a039 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.042180 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 55/107] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:56.042906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3bed9-b1aa-4079-9e8c-5e008770a039 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:56.111699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3c646-c467-49f1-ad1d-9fba5c8dde42 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:56.155232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4a327-e603-47d4-aa21-620ba9a850a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:56.156200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4a327-e603-47d4-aa21-620ba9a850a6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.156552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4a327-e603-47d4-aa21-620ba9a850a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.156815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4a327-e603-47d4-aa21-620ba9a850a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.165974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3bed9-b1aa-4079-9e8c-5e008770a039 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:56.168911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4a327-e603-47d4-aa21-620ba9a850a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ffe4a327-e603-47d4-aa21-620ba9a850a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:56.176810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:56.177686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:02:56.178512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 53/108] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:02:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:56.179591 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c2828-3112-4406-bd5f-902bca2d096d None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.179811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c2828-3112-4406-bd5f-902bca2d096d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.180220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c2828-3112-4406-bd5f-902bca2d096d None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.185924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 54/109] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:56.185924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a9122-2033-4953-8993-3143e076f3c9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.185924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a9122-2033-4953-8993-3143e076f3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.185924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a9122-2033-4953-8993-3143e076f3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.202007 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9e3a9122-2033-4953-8993-3143e076f3c9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:56.203463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9e3a9122-2033-4953-8993-3143e076f3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:56.210357 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad6d4e-3ea7-48ff-a00e-c4bbf5ff844c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:56.211291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad6d4e-3ea7-48ff-a00e-c4bbf5ff844c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.211652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad6d4e-3ea7-48ff-a00e-c4bbf5ff844c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.211934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad6d4e-3ea7-48ff-a00e-c4bbf5ff844c None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.213128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad6d4e-3ea7-48ff-a00e-c4bbf5ff844c None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:56.214626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad6d4e-3ea7-48ff-a00e-c4bbf5ff844c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:56.216313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 56/110] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:02:56 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:56.217718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad6d4e-3ea7-48ff-a00e-c4bbf5ff844c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:56.276861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:56.281373 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 55/111] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:56.282056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a9122-2033-4953-8993-3143e076f3c9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:56.309539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:56.341404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad6d4e-3ea7-48ff-a00e-c4bbf5ff844c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:56.376142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5143d-4a84-41cf-835c-80982f6b7c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:56.376944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5143d-4a84-41cf-835c-80982f6b7c16 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.377233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5143d-4a84-41cf-835c-80982f6b7c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.377503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5143d-4a84-41cf-835c-80982f6b7c16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.387900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5143d-4a84-41cf-835c-80982f6b7c16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:56.388597 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a9122-2033-4953-8993-3143e076f3c9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-06d5143d-4a84-41cf-835c-80982f6b7c16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:56.396055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:56.397136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:02:56.398033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 57/112] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:02:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:56.398734 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f443d-7501-4aa5-82c8-4820ed75490a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.398734 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f443d-7501-4aa5-82c8-4820ed75490a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.399004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f443d-7501-4aa5-82c8-4820ed75490a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.415964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-656f443d-7501-4aa5-82c8-4820ed75490a None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:56.417159 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-656f443d-7501-4aa5-82c8-4820ed75490a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:56.430370 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada0ffb-7e12-4a1f-9975-46a4a89aee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:56.431387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada0ffb-7e12-4a1f-9975-46a4a89aee6d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.431831 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada0ffb-7e12-4a1f-9975-46a4a89aee6d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.432295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada0ffb-7e12-4a1f-9975-46a4a89aee6d None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.433563 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada0ffb-7e12-4a1f-9975-46a4a89aee6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:56.434088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada0ffb-7e12-4a1f-9975-46a4a89aee6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:56.436531 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada0ffb-7e12-4a1f-9975-46a4a89aee6d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:56.449458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 56/113] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:02:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:56.450785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada0ffb-7e12-4a1f-9975-46a4a89aee6d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:56.497002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 58/114] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:56.498402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f443d-7501-4aa5-82c8-4820ed75490a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:56.537605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:56.542393 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:56.553695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada0ffb-7e12-4a1f-9975-46a4a89aee6d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:56.587996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a42dc-6375-430a-8cdd-36c5016c153a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:56.588889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a42dc-6375-430a-8cdd-36c5016c153a None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.589176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a42dc-6375-430a-8cdd-36c5016c153a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.589450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a42dc-6375-430a-8cdd-36c5016c153a None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477/users/331362e99eb14800b4d52472fd41a629/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.591468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a42dc-6375-430a-8cdd-36c5016c153a None admin] RBAC: Authorizing `identity:create_grant(project_id=f5b2e96e553d496d96e8f916fb522477, user_id=331362e99eb14800b4d52472fd41a629, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:56.593072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a42dc-6375-430a-8cdd-36c5016c153a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:56.605940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 57/115] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:02:56 2022] PUT /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477/users/331362e99eb14800b4d52472fd41a629/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:56.607217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5e49d-3182-474a-b7c7-f09d0d4a4fa2 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.607503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5e49d-3182-474a-b7c7-f09d0d4a4fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.607785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5e49d-3182-474a-b7c7-f09d0d4a4fa2 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.609131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 58/116] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:56.614232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5e49d-3182-474a-b7c7-f09d0d4a4fa2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:56.620632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f443d-7501-4aa5-82c8-4820ed75490a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:56.662513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03911b97-46a5-4301-a554-e79760d84b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:56.663342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03911b97-46a5-4301-a554-e79760d84b00 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.663624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03911b97-46a5-4301-a554-e79760d84b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.663902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03911b97-46a5-4301-a554-e79760d84b00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.665127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03911b97-46a5-4301-a554-e79760d84b00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:56.666709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03911b97-46a5-4301-a554-e79760d84b00 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:56.674326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 59/117] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:02:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:56.674326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03911b97-46a5-4301-a554-e79760d84b00 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:56.754239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5e49d-3182-474a-b7c7-f09d0d4a4fa2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:56.794216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03911b97-46a5-4301-a554-e79760d84b00 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:56.796000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:56.800885 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc21633-bd12-460a-93e5-b55daae28f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:56.801603 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc21633-bd12-460a-93e5-b55daae28f42 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.801923 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc21633-bd12-460a-93e5-b55daae28f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.802202 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc21633-bd12-460a-93e5-b55daae28f42 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.803265 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc21633-bd12-460a-93e5-b55daae28f42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:56.803580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc21633-bd12-460a-93e5-b55daae28f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:56.804977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc21633-bd12-460a-93e5-b55daae28f42 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:56.813867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 59/118] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:02:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:56.814878 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc21633-bd12-460a-93e5-b55daae28f42 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:56.826813 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:56.835895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c0289-d117-4675-82b8-d8596eb54859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:56.836767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c0289-d117-4675-82b8-d8596eb54859 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.837055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c0289-d117-4675-82b8-d8596eb54859 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.837328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c0289-d117-4675-82b8-d8596eb54859 None admin] PATH_INFO: `/v3/users/127a73b4627c49c6902e1cd23d25169e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.852032 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c0289-d117-4675-82b8-d8596eb54859 None admin] RBAC: Authorizing `identity:get_user(user_id=127a73b4627c49c6902e1cd23d25169e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:56.853974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c0289-d117-4675-82b8-d8596eb54859 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:56.856009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 60/119] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:56 2022] GET /identity/v3/users/127a73b4627c49c6902e1cd23d25169e => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:56.857305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f420b66-1ce9-4a89-9dbe-773c1cf20625 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.857819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f420b66-1ce9-4a89-9dbe-773c1cf20625 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.858109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f420b66-1ce9-4a89-9dbe-773c1cf20625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.874411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9f420b66-1ce9-4a89-9dbe-773c1cf20625 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:56.875599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9f420b66-1ce9-4a89-9dbe-773c1cf20625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:56.939436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc21633-bd12-460a-93e5-b55daae28f42 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:56.959244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 61/120] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:56.964930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f420b66-1ce9-4a89-9dbe-773c1cf20625 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:57.000558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc6957-3bd5-4ce5-b8dd-414dd99b6687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:57.001351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc6957-3bd5-4ce5-b8dd-414dd99b6687 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:57.001636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc6957-3bd5-4ce5-b8dd-414dd99b6687 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:57.001943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc6957-3bd5-4ce5-b8dd-414dd99b6687 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:57.014604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc6957-3bd5-4ce5-b8dd-414dd99b6687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-81dc6957-3bd5-4ce5-b8dd-414dd99b6687 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:57.022472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:57.023317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:02:57.024086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 60/121] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:02:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:57.025313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc6957-3bd5-4ce5-b8dd-414dd99b6687 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:57.063384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:57.074078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f420b66-1ce9-4a89-9dbe-773c1cf20625 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:57.098089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:57.114438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248585ef-07f5-443e-bb5b-4588c934d49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:57.115115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248585ef-07f5-443e-bb5b-4588c934d49b None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:57.115401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248585ef-07f5-443e-bb5b-4588c934d49b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:57.115677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248585ef-07f5-443e-bb5b-4588c934d49b None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:57.116393 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248585ef-07f5-443e-bb5b-4588c934d49b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:57.117883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248585ef-07f5-443e-bb5b-4588c934d49b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:57.140163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc6957-3bd5-4ce5-b8dd-414dd99b6687 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:57.153822 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 62/122] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:02:56 2022] POST /identity/v3/projects => generated 302 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:02:57.153822 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248585ef-07f5-443e-bb5b-4588c934d49b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:57.190677 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571645e-3a50-4435-8aa4-bf0f767875f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:57.191668 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571645e-3a50-4435-8aa4-bf0f767875f9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:57.192026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571645e-3a50-4435-8aa4-bf0f767875f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:57.192480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571645e-3a50-4435-8aa4-bf0f767875f9 None admin] PATH_INFO: `/v3/users/331362e99eb14800b4d52472fd41a629` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:57.195416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571645e-3a50-4435-8aa4-bf0f767875f9 None admin] RBAC: Authorizing `identity:get_user(user_id=331362e99eb14800b4d52472fd41a629)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:57.196999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571645e-3a50-4435-8aa4-bf0f767875f9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:57.198855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 61/123] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:57 2022] GET /identity/v3/users/331362e99eb14800b4d52472fd41a629 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:57.199777 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571645e-3a50-4435-8aa4-bf0f767875f9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:57.295035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248585ef-07f5-443e-bb5b-4588c934d49b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:57.333178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571645e-3a50-4435-8aa4-bf0f767875f9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:57.349660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee245d8-887b-45f3-9ed3-a41f766b732e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:57.350846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee245d8-887b-45f3-9ed3-a41f766b732e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:57.351463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee245d8-887b-45f3-9ed3-a41f766b732e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:57.352028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee245d8-887b-45f3-9ed3-a41f766b732e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:57.372730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ace237-da13-4cf6-aa22-ee8e94fba048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:57.374368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ace237-da13-4cf6-aa22-ee8e94fba048 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:57.374655 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ace237-da13-4cf6-aa22-ee8e94fba048 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:57.374988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ace237-da13-4cf6-aa22-ee8e94fba048 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:57.375408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:57.378812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:57.392387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ace237-da13-4cf6-aa22-ee8e94fba048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e8ace237-da13-4cf6-aa22-ee8e94fba048 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:57.400461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:57.402669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 62/124] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:02:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:57.403731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ace237-da13-4cf6-aa22-ee8e94fba048 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:57.452714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee245d8-887b-45f3-9ed3-a41f766b732e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:57.454642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee245d8-887b-45f3-9ed3-a41f766b732e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:57.525751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ace237-da13-4cf6-aa22-ee8e94fba048 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:57.565876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257e93a-904b-437d-98bc-304d90c98802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:57.567078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257e93a-904b-437d-98bc-304d90c98802 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:57.567362 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257e93a-904b-437d-98bc-304d90c98802 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:57.567635 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257e93a-904b-437d-98bc-304d90c98802 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:57.568668 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257e93a-904b-437d-98bc-304d90c98802 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:57.568980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257e93a-904b-437d-98bc-304d90c98802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:57.571042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257e93a-904b-437d-98bc-304d90c98802 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:57.575083 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 63/125] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:02:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:02:57.576827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee245d8-887b-45f3-9ed3-a41f766b732e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:57.586388 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 63/126] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:57.587090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257e93a-904b-437d-98bc-304d90c98802 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:57.641489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:57.679991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:57.705226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee245d8-887b-45f3-9ed3-a41f766b732e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:57.750400 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257e93a-904b-437d-98bc-304d90c98802 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:57.761495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d17819-6447-431b-9169-ea14b47ab01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:57.763339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d17819-6447-431b-9169-ea14b47ab01f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:57.763851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d17819-6447-431b-9169-ea14b47ab01f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:57.764311 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d17819-6447-431b-9169-ea14b47ab01f None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:57.766567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d17819-6447-431b-9169-ea14b47ab01f None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:57.768775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d17819-6447-431b-9169-ea14b47ab01f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:57.771134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 64/127] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:02:57 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:57.772263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d17819-6447-431b-9169-ea14b47ab01f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:57.795499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4808112-a373-4a34-a923-96b0b24469be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:57.798286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4808112-a373-4a34-a923-96b0b24469be None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:57.798574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4808112-a373-4a34-a923-96b0b24469be None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:57.799111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4808112-a373-4a34-a923-96b0b24469be None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:57.799875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4808112-a373-4a34-a923-96b0b24469be None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:57.800185 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4808112-a373-4a34-a923-96b0b24469be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:57.802062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4808112-a373-4a34-a923-96b0b24469be None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:57.813852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 64/128] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:02:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:57.815270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4808112-a373-4a34-a923-96b0b24469be None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:57.896708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d17819-6447-431b-9169-ea14b47ab01f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:57.910178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:57.934876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb1f57-2da6-4de4-9bed-aad231a1228e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:57.935767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb1f57-2da6-4de4-9bed-aad231a1228e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:57.936157 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb1f57-2da6-4de4-9bed-aad231a1228e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:57.936439 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb1f57-2da6-4de4-9bed-aad231a1228e None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:57.937717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb1f57-2da6-4de4-9bed-aad231a1228e None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:57.939209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb1f57-2da6-4de4-9bed-aad231a1228e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:57.939986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4808112-a373-4a34-a923-96b0b24469be None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:57.940981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 65/129] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:02:57 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:57.941919 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb1f57-2da6-4de4-9bed-aad231a1228e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:57.965095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:57.982995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e9938-77a2-40e6-aa4f-d8037e570c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:57.983878 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e9938-77a2-40e6-aa4f-d8037e570c91 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:57.984166 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e9938-77a2-40e6-aa4f-d8037e570c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:57.984444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e9938-77a2-40e6-aa4f-d8037e570c91 None admin] PATH_INFO: `/v3/users/3b98d845867d491c91aeca1e12cf3ace` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:57.986494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e9938-77a2-40e6-aa4f-d8037e570c91 None admin] RBAC: Authorizing `identity:get_user(user_id=3b98d845867d491c91aeca1e12cf3ace)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:57.988161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e9938-77a2-40e6-aa4f-d8037e570c91 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:57.990560 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 65/130] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:57 2022] GET /identity/v3/users/3b98d845867d491c91aeca1e12cf3ace => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:57.995297 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e9938-77a2-40e6-aa4f-d8037e570c91 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:58.069490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb1f57-2da6-4de4-9bed-aad231a1228e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:58.108148 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c8d15-e9d5-47dd-a591-591e822827d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:58.109264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c8d15-e9d5-47dd-a591-591e822827d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:58.109608 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c8d15-e9d5-47dd-a591-591e822827d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:58.109872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c8d15-e9d5-47dd-a591-591e822827d8 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477/users/331362e99eb14800b4d52472fd41a629/roles/506b3cf33324478abe0f72a9d408aa54` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:58.112307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c8d15-e9d5-47dd-a591-591e822827d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5b2e96e553d496d96e8f916fb522477, user_id=331362e99eb14800b4d52472fd41a629, role_id=506b3cf33324478abe0f72a9d408aa54)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:58.113878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c8d15-e9d5-47dd-a591-591e822827d8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:58.119157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e9938-77a2-40e6-aa4f-d8037e570c91 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:58.133027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 66/131] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:02:57 2022] PUT /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477/users/331362e99eb14800b4d52472fd41a629/roles/506b3cf33324478abe0f72a9d408aa54 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:58.133838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c8d15-e9d5-47dd-a591-591e822827d8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:58.183397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:58.186654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5646324-51a4-4cca-8d73-c79eaca0a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:58.189004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5646324-51a4-4cca-8d73-c79eaca0a382 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:58.189484 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5646324-51a4-4cca-8d73-c79eaca0a382 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:58.189966 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5646324-51a4-4cca-8d73-c79eaca0a382 None admin] PATH_INFO: `/v3/users/331362e99eb14800b4d52472fd41a629` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:58.191527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5646324-51a4-4cca-8d73-c79eaca0a382 None admin] RBAC: Authorizing `identity:get_user(user_id=331362e99eb14800b4d52472fd41a629)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:58.193207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5646324-51a4-4cca-8d73-c79eaca0a382 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:58.201756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 66/132] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:57 2022] GET /identity/v3/users/331362e99eb14800b4d52472fd41a629 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:58.203830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5646324-51a4-4cca-8d73-c79eaca0a382 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:58.257170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c8d15-e9d5-47dd-a591-591e822827d8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:58.297007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30ce45-501e-4556-9223-e0d05167be01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:58.297778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30ce45-501e-4556-9223-e0d05167be01 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:58.297996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30ce45-501e-4556-9223-e0d05167be01 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:58.298278 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30ce45-501e-4556-9223-e0d05167be01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:58.299244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30ce45-501e-4556-9223-e0d05167be01 None admin] RBAC: Adding query filter params (role.id=45c97834bc614c5ca5122d691ce93316, scope.project.id=f5b2e96e553d496d96e8f916fb522477, user.id=127a73b4627c49c6902e1cd23d25169e) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:58.299660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30ce45-501e-4556-9223-e0d05167be01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:58.301292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30ce45-501e-4556-9223-e0d05167be01 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:58.308120 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 67/133] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:02:58 2022] GET /identity/v3/role_assignments?user.id=127a73b4627c49c6902e1cd23d25169e&scope.project.id=f5b2e96e553d496d96e8f916fb522477&role.id=45c97834bc614c5ca5122d691ce93316 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:58.308998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30ce45-501e-4556-9223-e0d05167be01 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:58.393060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:58.481908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5646324-51a4-4cca-8d73-c79eaca0a382 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:58.528251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30ce45-501e-4556-9223-e0d05167be01 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:58.531880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:58.555386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16675956-6403-4266-8a75-d65753727854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:58.556262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16675956-6403-4266-8a75-d65753727854 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:58.556541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16675956-6403-4266-8a75-d65753727854 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:58.556810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16675956-6403-4266-8a75-d65753727854 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:58.558882 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16675956-6403-4266-8a75-d65753727854 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:58.560565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16675956-6403-4266-8a75-d65753727854 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:58.564308 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 67/134] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:02:58 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:58.570441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16675956-6403-4266-8a75-d65753727854 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:58.635054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e8af2-4b3b-445e-9a39-8e492e87079f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:58.635935 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e8af2-4b3b-445e-9a39-8e492e87079f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:58.636221 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e8af2-4b3b-445e-9a39-8e492e87079f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:58.636491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e8af2-4b3b-445e-9a39-8e492e87079f None admin] PATH_INFO: `/v3/projects/db9fa468925d45d9901c0bed2034e05f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:58.642517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e8af2-4b3b-445e-9a39-8e492e87079f None admin] RBAC: Authorizing `identity:get_project(project_id=db9fa468925d45d9901c0bed2034e05f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:58.644012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e8af2-4b3b-445e-9a39-8e492e87079f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:58.645761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 68/135] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:02:58 2022] GET /identity/v3/projects/db9fa468925d45d9901c0bed2034e05f => generated 312 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:58.651324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e8af2-4b3b-445e-9a39-8e492e87079f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:58.733369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16675956-6403-4266-8a75-d65753727854 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:58.780242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:58.784048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e8af2-4b3b-445e-9a39-8e492e87079f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:58.804251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eebef1-953a-4b34-9697-eb2700cc2082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:58.805133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eebef1-953a-4b34-9697-eb2700cc2082 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:58.805417 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eebef1-953a-4b34-9697-eb2700cc2082 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:58.805691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eebef1-953a-4b34-9697-eb2700cc2082 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477/users/3b98d845867d491c91aeca1e12cf3ace/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:58.807966 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eebef1-953a-4b34-9697-eb2700cc2082 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5b2e96e553d496d96e8f916fb522477, user_id=3b98d845867d491c91aeca1e12cf3ace, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:58.809495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eebef1-953a-4b34-9697-eb2700cc2082 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:58.828801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346036a-824c-40f3-93f5-1a67c0c0257e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:58.830248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346036a-824c-40f3-93f5-1a67c0c0257e None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:58.830769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346036a-824c-40f3-93f5-1a67c0c0257e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:58.831277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346036a-824c-40f3-93f5-1a67c0c0257e None admin] PATH_INFO: `/v3/projects/db9fa468925d45d9901c0bed2034e05f/users/331362e99eb14800b4d52472fd41a629/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:58.833769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346036a-824c-40f3-93f5-1a67c0c0257e None admin] RBAC: Authorizing `identity:create_grant(project_id=db9fa468925d45d9901c0bed2034e05f, user_id=331362e99eb14800b4d52472fd41a629, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:58.835592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346036a-824c-40f3-93f5-1a67c0c0257e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:58.850056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 68/136] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:02:58 2022] PUT /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477/users/3b98d845867d491c91aeca1e12cf3ace/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:58.853852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 69/137] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:02:58 2022] PUT /identity/v3/projects/db9fa468925d45d9901c0bed2034e05f/users/331362e99eb14800b4d52472fd41a629/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:02:59.028063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d45e3-b2a0-4215-a577-71d387eedbfb None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:59.028305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d45e3-b2a0-4215-a577-71d387eedbfb None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:59.028701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d45e3-b2a0-4215-a577-71d387eedbfb None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:59.030582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 69/138] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:02:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:59.036911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb83e6-5285-47bd-885b-7d7f5168abef None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:59.037202 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb83e6-5285-47bd-885b-7d7f5168abef None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:59.037468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb83e6-5285-47bd-885b-7d7f5168abef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:59.062016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-21fb83e6-5285-47bd-885b-7d7f5168abef None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:59.063230 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-21fb83e6-5285-47bd-885b-7d7f5168abef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:59.111546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:59.172693 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 70/139] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:02:59.197085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d45e3-b2a0-4215-a577-71d387eedbfb None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:59.321054 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d45e3-b2a0-4215-a577-71d387eedbfb None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:59.362356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552c30a-2b3e-4d0b-adfb-c5bff2c0cc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:59.363161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552c30a-2b3e-4d0b-adfb-c5bff2c0cc4e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:59.363445 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552c30a-2b3e-4d0b-adfb-c5bff2c0cc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:59.363716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552c30a-2b3e-4d0b-adfb-c5bff2c0cc4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:59.381567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552c30a-2b3e-4d0b-adfb-c5bff2c0cc4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-6552c30a-2b3e-4d0b-adfb-c5bff2c0cc4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:02:59.390977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:02:59.391861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:02:59.392619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 70/140] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:02:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:02:59.398861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb83e6-5285-47bd-885b-7d7f5168abef None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:59.530672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb83e6-5285-47bd-885b-7d7f5168abef None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:59.575456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9567b91-e82a-4262-bfb3-0d0a1269c621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:59.576097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9567b91-e82a-4262-bfb3-0d0a1269c621 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:59.576389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9567b91-e82a-4262-bfb3-0d0a1269c621 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:59.576671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9567b91-e82a-4262-bfb3-0d0a1269c621 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:59.577542 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9567b91-e82a-4262-bfb3-0d0a1269c621 None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:59.577884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9567b91-e82a-4262-bfb3-0d0a1269c621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:59.579299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9567b91-e82a-4262-bfb3-0d0a1269c621 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:59.594211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 71/141] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:02:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:02:59.599835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552c30a-2b3e-4d0b-adfb-c5bff2c0cc4e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:59.683991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:59.750295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552c30a-2b3e-4d0b-adfb-c5bff2c0cc4e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:59.805817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5165a3d-ae80-4910-b475-dda248173f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:59.806746 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5165a3d-ae80-4910-b475-dda248173f70 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:59.807093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5165a3d-ae80-4910-b475-dda248173f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:59.807365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5165a3d-ae80-4910-b475-dda248173f70 None admin] PATH_INFO: `/v3/users/331362e99eb14800b4d52472fd41a629` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:59.808612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5165a3d-ae80-4910-b475-dda248173f70 None admin] RBAC: Authorizing `identity:get_user(user_id=331362e99eb14800b4d52472fd41a629)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:59.810103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5165a3d-ae80-4910-b475-dda248173f70 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:59.812884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 71/142] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:02:59 2022] GET /identity/v3/users/331362e99eb14800b4d52472fd41a629 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:59.818885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9567b91-e82a-4262-bfb3-0d0a1269c621 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:59.851714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:59.955920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9567b91-e82a-4262-bfb3-0d0a1269c621 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:59.992132 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b10b9-01e9-4329-9cbf-64538dc85179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:59.992788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b10b9-01e9-4329-9cbf-64538dc85179 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:59.993066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b10b9-01e9-4329-9cbf-64538dc85179 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:59.993341 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b10b9-01e9-4329-9cbf-64538dc85179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:00.068095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b10b9-01e9-4329-9cbf-64538dc85179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:00.069595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b10b9-01e9-4329-9cbf-64538dc85179 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:00.095524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:00.195661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 72/143] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:02:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:00.202230 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5165a3d-ae80-4910-b475-dda248173f70 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:00.333041 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5165a3d-ae80-4910-b475-dda248173f70 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:00.347989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:00.382812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1071d0-d775-46c2-a70b-4fa56b6efdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:00.383905 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1071d0-d775-46c2-a70b-4fa56b6efdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:00.384323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1071d0-d775-46c2-a70b-4fa56b6efdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:00.384763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1071d0-d775-46c2-a70b-4fa56b6efdb3 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:00.390554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1071d0-d775-46c2-a70b-4fa56b6efdb3 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:00.392163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1071d0-d775-46c2-a70b-4fa56b6efdb3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:00.394465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 72/144] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:00 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:00.400977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b10b9-01e9-4329-9cbf-64538dc85179 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:00.530445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b10b9-01e9-4329-9cbf-64538dc85179 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:00.554243 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:00.570444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abdd67-ef39-4e22-b117-c6c467175fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:00.571104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abdd67-ef39-4e22-b117-c6c467175fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:00.571387 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abdd67-ef39-4e22-b117-c6c467175fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:00.571659 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abdd67-ef39-4e22-b117-c6c467175fd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:00.572490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abdd67-ef39-4e22-b117-c6c467175fd5 None admin] RBAC: Adding query filter params (role.id=e1a5192e7abc4046bc2c0673176ea1b6, scope.project.id=f5b2e96e553d496d96e8f916fb522477, user.id=331362e99eb14800b4d52472fd41a629) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:00.572799 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abdd67-ef39-4e22-b117-c6c467175fd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:00.574299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abdd67-ef39-4e22-b117-c6c467175fd5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:00.581451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 73/145] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:00 2022] GET /identity/v3/role_assignments?user.id=331362e99eb14800b4d52472fd41a629&scope.project.id=f5b2e96e553d496d96e8f916fb522477&role.id=e1a5192e7abc4046bc2c0673176ea1b6 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:00.596335 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366f577-292d-4b42-af6a-45354b1d13ef None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:00.597004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366f577-292d-4b42-af6a-45354b1d13ef None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:00.597468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366f577-292d-4b42-af6a-45354b1d13ef None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:00.599357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 73/146] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:00.604499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ab3c0-9107-4d55-93e0-1a3fb0116525 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:00.604791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ab3c0-9107-4d55-93e0-1a3fb0116525 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:00.605071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ab3c0-9107-4d55-93e0-1a3fb0116525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:00.627605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ca4ab3c0-9107-4d55-93e0-1a3fb0116525 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:00.628829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ca4ab3c0-9107-4d55-93e0-1a3fb0116525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:00.720693 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 74/147] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:00.757902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366f577-292d-4b42-af6a-45354b1d13ef None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:00.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366f577-292d-4b42-af6a-45354b1d13ef None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:00.937824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ca8e7-7eb7-4bdd-965f-b74b51a79b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:00.938995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ca8e7-7eb7-4bdd-965f-b74b51a79b3e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:00.939479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ca8e7-7eb7-4bdd-965f-b74b51a79b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:00.939942 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ca8e7-7eb7-4bdd-965f-b74b51a79b3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:00.941762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ca8e7-7eb7-4bdd-965f-b74b51a79b3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:00.943764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ca8e7-7eb7-4bdd-965f-b74b51a79b3e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:00.946026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 74/148] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:00.954968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ab3c0-9107-4d55-93e0-1a3fb0116525 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:01.076640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ab3c0-9107-4d55-93e0-1a3fb0116525 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:01.100870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:01.130523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262aeb7a-a03e-4da0-a6a2-c82acaf2c606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:01.131222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262aeb7a-a03e-4da0-a6a2-c82acaf2c606 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.131505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262aeb7a-a03e-4da0-a6a2-c82acaf2c606 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.131779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262aeb7a-a03e-4da0-a6a2-c82acaf2c606 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.132499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262aeb7a-a03e-4da0-a6a2-c82acaf2c606 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:01.134003 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262aeb7a-a03e-4da0-a6a2-c82acaf2c606 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:01.174234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 75/149] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:03:00 2022] POST /identity/v3/users => generated 283 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:01.554329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6af7a-d103-43cd-ba8c-04996277afe1 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.554791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6af7a-d103-43cd-ba8c-04996277afe1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.554791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6af7a-d103-43cd-ba8c-04996277afe1 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.556317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 75/150] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:01.563328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab1cdc-3c88-4984-abe9-1227f804e615 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.563765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab1cdc-3c88-4984-abe9-1227f804e615 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.563945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab1cdc-3c88-4984-abe9-1227f804e615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.582235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1bab1cdc-3c88-4984-abe9-1227f804e615 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:01.583444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1bab1cdc-3c88-4984-abe9-1227f804e615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:01.665892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 76/151] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:01.684828 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e6af7a-d103-43cd-ba8c-04996277afe1 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:01.743722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:01.815172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e6af7a-d103-43cd-ba8c-04996277afe1 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:01.858240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d2e39-19c6-48e5-b5a6-3252da8a2006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:01.859064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d2e39-19c6-48e5-b5a6-3252da8a2006 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.859346 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d2e39-19c6-48e5-b5a6-3252da8a2006 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.859616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d2e39-19c6-48e5-b5a6-3252da8a2006 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.867268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52304767-ae41-48bb-8654-2fb3693139f1 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.867759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52304767-ae41-48bb-8654-2fb3693139f1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.867863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52304767-ae41-48bb-8654-2fb3693139f1 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.869574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 77/152] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:01.874084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d2e39-19c6-48e5-b5a6-3252da8a2006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-937d2e39-19c6-48e5-b5a6-3252da8a2006 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:01.883749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:01.884608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:01.885368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 76/153] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:01.890868 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728fbc0-7069-441b-91b0-b330674a5cfd None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.891201 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728fbc0-7069-441b-91b0-b330674a5cfd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.891486 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728fbc0-7069-441b-91b0-b330674a5cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.895405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d2e39-19c6-48e5-b5a6-3252da8a2006 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:01.910813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e728fbc0-7069-441b-91b0-b330674a5cfd None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:01.912321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e728fbc0-7069-441b-91b0-b330674a5cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:02.004685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 78/154] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:02.023462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d2e39-19c6-48e5-b5a6-3252da8a2006 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:02.035096 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:02.052362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728fbc0-7069-441b-91b0-b330674a5cfd None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:02.079884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbac2f-1cd9-489e-a1f8-f7d212389ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:02.081091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbac2f-1cd9-489e-a1f8-f7d212389ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.082039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbac2f-1cd9-489e-a1f8-f7d212389ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.082573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbac2f-1cd9-489e-a1f8-f7d212389ec5 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.083580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cbac2f-1cd9-489e-a1f8-f7d212389ec5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:02.084103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cbac2f-1cd9-489e-a1f8-f7d212389ec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:02.086697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cbac2f-1cd9-489e-a1f8-f7d212389ec5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:02.099953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:02.100558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 77/155] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:02.110852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbac2f-1cd9-489e-a1f8-f7d212389ec5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:02.184936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728fbc0-7069-441b-91b0-b330674a5cfd None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:02.224451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999c788-5245-4e43-a851-b8d710b42a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:02.225507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999c788-5245-4e43-a851-b8d710b42a6c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.225964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999c788-5245-4e43-a851-b8d710b42a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.226407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999c788-5245-4e43-a851-b8d710b42a6c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.227387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbac2f-1cd9-489e-a1f8-f7d212389ec5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:02.250386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999c788-5245-4e43-a851-b8d710b42a6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-7999c788-5245-4e43-a851-b8d710b42a6c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:02.258853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:02.259915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:02.260757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:02.260757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:02.260757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:02.260757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:02.260757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:02.260757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:02.260757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:02.260757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:02.260757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:02.260757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:02.260757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:02.260757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:02.263590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 79/156] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:03:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:02.270139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999c788-5245-4e43-a851-b8d710b42a6c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:02.270431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acbd13-ed62-44dd-b147-260507b3793d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:02.271368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acbd13-ed62-44dd-b147-260507b3793d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.271638 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acbd13-ed62-44dd-b147-260507b3793d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.271906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acbd13-ed62-44dd-b147-260507b3793d None admin] PATH_INFO: `/v3/users/127a73b4627c49c6902e1cd23d25169e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.273105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3acbd13-ed62-44dd-b147-260507b3793d None admin] RBAC: Authorizing `identity:get_user(user_id=127a73b4627c49c6902e1cd23d25169e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:02.277754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3acbd13-ed62-44dd-b147-260507b3793d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:02.280890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 78/157] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:02 2022] GET /identity/v3/users/127a73b4627c49c6902e1cd23d25169e => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:02.284425 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acbd13-ed62-44dd-b147-260507b3793d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:02.331855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:02.408970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acbd13-ed62-44dd-b147-260507b3793d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:02.432175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:02.441668 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999c788-5245-4e43-a851-b8d710b42a6c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:02.452421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0fb79-7a99-4d4b-88a0-de1c5e4ea39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:02.453685 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0fb79-7a99-4d4b-88a0-de1c5e4ea39a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.454015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0fb79-7a99-4d4b-88a0-de1c5e4ea39a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.454288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0fb79-7a99-4d4b-88a0-de1c5e4ea39a None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.456413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0fb79-7a99-4d4b-88a0-de1c5e4ea39a None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:02.458328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0fb79-7a99-4d4b-88a0-de1c5e4ea39a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:02.460669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 79/158] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:02 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:02.462263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ca72-ac47-425e-a130-a01659a12236 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.462703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ca72-ac47-425e-a130-a01659a12236 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.462983 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ca72-ac47-425e-a130-a01659a12236 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.464746 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 80/159] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:02.465684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dba7a-c4e7-4946-a533-4f5a4407a1ea None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.465991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dba7a-c4e7-4946-a533-4f5a4407a1ea None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.467300 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dba7a-c4e7-4946-a533-4f5a4407a1ea None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.468852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 81/160] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:02.469568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dba7a-c4e7-4946-a533-4f5a4407a1ea None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:02.488777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716c0f7-80f9-4055-a000-03839159bbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:02.489509 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716c0f7-80f9-4055-a000-03839159bbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.489926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716c0f7-80f9-4055-a000-03839159bbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.490206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716c0f7-80f9-4055-a000-03839159bbe5 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.490953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716c0f7-80f9-4055-a000-03839159bbe5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:02.491377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716c0f7-80f9-4055-a000-03839159bbe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:02.492780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716c0f7-80f9-4055-a000-03839159bbe5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:02.503742 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 80/161] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:02.505161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ddf0d-1e5d-48fd-a1d5-997bb9bdebf5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.505448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ddf0d-1e5d-48fd-a1d5-997bb9bdebf5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.505756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ddf0d-1e5d-48fd-a1d5-997bb9bdebf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.525111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ad6ddf0d-1e5d-48fd-a1d5-997bb9bdebf5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:02.525950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ad6ddf0d-1e5d-48fd-a1d5-997bb9bdebf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:02.601129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dba7a-c4e7-4946-a533-4f5a4407a1ea None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:02.607980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 81/162] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:02.609294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17985334-c466-49c1-9b33-9e69df244eb7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.609578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17985334-c466-49c1-9b33-9e69df244eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.609901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17985334-c466-49c1-9b33-9e69df244eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.617766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:02.646646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-17985334-c466-49c1-9b33-9e69df244eb7 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:02.647835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-17985334-c466-49c1-9b33-9e69df244eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:02.657113 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815517a-8564-47f6-b8cc-9695b103c5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:02.658859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815517a-8564-47f6-b8cc-9695b103c5e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.659359 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815517a-8564-47f6-b8cc-9695b103c5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.660494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815517a-8564-47f6-b8cc-9695b103c5e9 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477/users/127a73b4627c49c6902e1cd23d25169e/roles/45c97834bc614c5ca5122d691ce93316` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.663851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815517a-8564-47f6-b8cc-9695b103c5e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5b2e96e553d496d96e8f916fb522477, user_id=127a73b4627c49c6902e1cd23d25169e, role_id=45c97834bc614c5ca5122d691ce93316)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:02.666192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815517a-8564-47f6-b8cc-9695b103c5e9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:02.699378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 82/163] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:03:02 2022] PUT /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477/users/127a73b4627c49c6902e1cd23d25169e/roles/45c97834bc614c5ca5122d691ce93316 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:02.703258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815517a-8564-47f6-b8cc-9695b103c5e9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:02.772669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 82/164] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:02.775029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17985334-c466-49c1-9b33-9e69df244eb7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:02.811788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:02.916918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815517a-8564-47f6-b8cc-9695b103c5e9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:02.940751 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17985334-c466-49c1-9b33-9e69df244eb7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:02.979550 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbfc98-1555-415d-8128-6768609f0ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:02.980437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbfc98-1555-415d-8128-6768609f0ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.980722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbfc98-1555-415d-8128-6768609f0ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.980998 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbfc98-1555-415d-8128-6768609f0ff4 None admin] PATH_INFO: `/v3/users/331362e99eb14800b4d52472fd41a629` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.984898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea4a60-276b-4e27-80b5-e0e6a4444fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:02.984898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea4a60-276b-4e27-80b5-e0e6a4444fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.984898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea4a60-276b-4e27-80b5-e0e6a4444fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.984898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea4a60-276b-4e27-80b5-e0e6a4444fb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.986513 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbfc98-1555-415d-8128-6768609f0ff4 None admin] RBAC: Authorizing `identity:get_user(user_id=331362e99eb14800b4d52472fd41a629)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:02.988009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbfc98-1555-415d-8128-6768609f0ff4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:02.989796 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 83/165] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:02 2022] GET /identity/v3/users/331362e99eb14800b4d52472fd41a629 => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:02.990594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbfc98-1555-415d-8128-6768609f0ff4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:02.997562 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea4a60-276b-4e27-80b5-e0e6a4444fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c5ea4a60-276b-4e27-80b5-e0e6a4444fb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:03.006733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:03.007656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:03.008419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:03.008419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:03.008419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:03.008419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:03.008419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:03.008419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:03.008419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:03.008419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:03.008419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:03.008419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:03.008419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:03.008419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:03.011065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 83/166] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:03:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:03.012177 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea4a60-276b-4e27-80b5-e0e6a4444fb6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:03.050539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:03.130243 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:03.149040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbfc98-1555-415d-8128-6768609f0ff4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:03.180374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea4a60-276b-4e27-80b5-e0e6a4444fb6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:03.186112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae9327-4d9c-4526-8c42-94a9c163710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:03.186907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae9327-4d9c-4526-8c42-94a9c163710a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:03.187190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae9327-4d9c-4526-8c42-94a9c163710a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:03.187467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae9327-4d9c-4526-8c42-94a9c163710a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:03.198360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae9327-4d9c-4526-8c42-94a9c163710a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e0ae9327-4d9c-4526-8c42-94a9c163710a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:03.206565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:03.207495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:03.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 84/167] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:03.208891 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae9327-4d9c-4526-8c42-94a9c163710a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:03.262669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cdfb4-2e34-4e2a-8ff9-98ee4335d801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:03.263995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cdfb4-2e34-4e2a-8ff9-98ee4335d801 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:03.264393 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cdfb4-2e34-4e2a-8ff9-98ee4335d801 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:03.264815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cdfb4-2e34-4e2a-8ff9-98ee4335d801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:03.322043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:03.326676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae9327-4d9c-4526-8c42-94a9c163710a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:03.362977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd63e6-77b3-4698-a067-8b060774bc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:03.363751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd63e6-77b3-4698-a067-8b060774bc57 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:03.364123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd63e6-77b3-4698-a067-8b060774bc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:03.364453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd63e6-77b3-4698-a067-8b060774bc57 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:03.365273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd63e6-77b3-4698-a067-8b060774bc57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:03.365658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd63e6-77b3-4698-a067-8b060774bc57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:03.367139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd63e6-77b3-4698-a067-8b060774bc57 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:03.376863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 85/168] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:03.377652 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd63e6-77b3-4698-a067-8b060774bc57 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:03.467705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cdfb4-2e34-4e2a-8ff9-98ee4335d801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:03.469297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cdfb4-2e34-4e2a-8ff9-98ee4335d801 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:03.489962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd63e6-77b3-4698-a067-8b060774bc57 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:03.528287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773cf36-e66d-4764-b53e-4aca5de84394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:03.529833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773cf36-e66d-4764-b53e-4aca5de84394 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:03.530308 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773cf36-e66d-4764-b53e-4aca5de84394 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:03.530606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773cf36-e66d-4764-b53e-4aca5de84394 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:03.531391 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773cf36-e66d-4764-b53e-4aca5de84394 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:03.531701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773cf36-e66d-4764-b53e-4aca5de84394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:03.533117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773cf36-e66d-4764-b53e-4aca5de84394 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:03.542777 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 86/169] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:03.543784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773cf36-e66d-4764-b53e-4aca5de84394 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:03.572656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:03.655281 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773cf36-e66d-4764-b53e-4aca5de84394 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:03.692851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea08eb-3f90-405f-b139-0fe7635d487a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:03.693722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea08eb-3f90-405f-b139-0fe7635d487a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:03.694033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea08eb-3f90-405f-b139-0fe7635d487a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:03.694308 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea08eb-3f90-405f-b139-0fe7635d487a None admin] PATH_INFO: `/v3/users/3b98d845867d491c91aeca1e12cf3ace` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:03.695561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea08eb-3f90-405f-b139-0fe7635d487a None admin] RBAC: Authorizing `identity:get_user(user_id=3b98d845867d491c91aeca1e12cf3ace)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:03.697009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea08eb-3f90-405f-b139-0fe7635d487a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:03.698665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 87/170] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:03 2022] GET /identity/v3/users/3b98d845867d491c91aeca1e12cf3ace => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:03.699435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea08eb-3f90-405f-b139-0fe7635d487a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:03.712197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:03.792942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 84/171] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 781 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:03.793759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cdfb4-2e34-4e2a-8ff9-98ee4335d801 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:03.797224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:03.809496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea08eb-3f90-405f-b139-0fe7635d487a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:03.849467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af32a7-755d-4994-b5e7-8b22d7403d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:03.850376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af32a7-755d-4994-b5e7-8b22d7403d3e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:03.850659 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af32a7-755d-4994-b5e7-8b22d7403d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:03.850930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af32a7-755d-4994-b5e7-8b22d7403d3e None admin] PATH_INFO: `/v3/users/331362e99eb14800b4d52472fd41a629` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:03.852201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af32a7-755d-4994-b5e7-8b22d7403d3e None admin] RBAC: Authorizing `identity:get_user(user_id=331362e99eb14800b4d52472fd41a629)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:03.853691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af32a7-755d-4994-b5e7-8b22d7403d3e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:03.855790 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 88/172] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:03 2022] GET /identity/v3/users/331362e99eb14800b4d52472fd41a629 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:03.856562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af32a7-755d-4994-b5e7-8b22d7403d3e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:03.992520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af32a7-755d-4994-b5e7-8b22d7403d3e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:04.030959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0796bd-cf96-4b8c-875d-fd82be41f9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:04.031821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0796bd-cf96-4b8c-875d-fd82be41f9bc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:04.032103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0796bd-cf96-4b8c-875d-fd82be41f9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:04.032661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0796bd-cf96-4b8c-875d-fd82be41f9bc None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:04.033962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0796bd-cf96-4b8c-875d-fd82be41f9bc None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:04.036312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0796bd-cf96-4b8c-875d-fd82be41f9bc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:04.038120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 89/173] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:03 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:04.039095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0796bd-cf96-4b8c-875d-fd82be41f9bc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:04.090222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:04.156539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0796bd-cf96-4b8c-875d-fd82be41f9bc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:04.159879 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cdfb4-2e34-4e2a-8ff9-98ee4335d801 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:04.210097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1d26a-2013-438a-aea1-0ae8e177c150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:04.210758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1d26a-2013-438a-aea1-0ae8e177c150 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:04.211066 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1d26a-2013-438a-aea1-0ae8e177c150 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:04.211603 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1d26a-2013-438a-aea1-0ae8e177c150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:04.272089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01333bf7-8271-4154-85c7-db7579c52360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:04.272089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01333bf7-8271-4154-85c7-db7579c52360 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:04.272089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01333bf7-8271-4154-85c7-db7579c52360 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:04.272089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01333bf7-8271-4154-85c7-db7579c52360 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:04.299183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1d26a-2013-438a-aea1-0ae8e177c150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:04.306852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1d26a-2013-438a-aea1-0ae8e177c150 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:04.354200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:04.369563 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:04.396324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01333bf7-8271-4154-85c7-db7579c52360 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:04.397850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01333bf7-8271-4154-85c7-db7579c52360 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:04.501465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 85/174] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 708 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:04.502600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01333bf7-8271-4154-85c7-db7579c52360 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:04.518422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 90/175] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:04.519312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1d26a-2013-438a-aea1-0ae8e177c150 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:04.622167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:04.636493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1d26a-2013-438a-aea1-0ae8e177c150 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:04.658546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01333bf7-8271-4154-85c7-db7579c52360 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:04.666642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:04.681470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756eee0-1f56-4d90-9454-e731b06091a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:04.682551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756eee0-1f56-4d90-9454-e731b06091a0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:04.682873 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756eee0-1f56-4d90-9454-e731b06091a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:04.683205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756eee0-1f56-4d90-9454-e731b06091a0 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:04.684655 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756eee0-1f56-4d90-9454-e731b06091a0 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:04.686868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756eee0-1f56-4d90-9454-e731b06091a0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:04.688766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 91/176] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:04 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:04.689451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756eee0-1f56-4d90-9454-e731b06091a0 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:04.722444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49677e-ba56-47b0-b95a-1b70de38dc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:04.723115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49677e-ba56-47b0-b95a-1b70de38dc21 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:04.723384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49677e-ba56-47b0-b95a-1b70de38dc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:04.723695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49677e-ba56-47b0-b95a-1b70de38dc21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:04.724494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49677e-ba56-47b0-b95a-1b70de38dc21 None admin] RBAC: Adding query filter params (role.id=506b3cf33324478abe0f72a9d408aa54, scope.project.id=f5b2e96e553d496d96e8f916fb522477, user.id=331362e99eb14800b4d52472fd41a629) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:04.724806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49677e-ba56-47b0-b95a-1b70de38dc21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:04.726323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49677e-ba56-47b0-b95a-1b70de38dc21 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:04.736750 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 86/177] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:04 2022] GET /identity/v3/role_assignments?user.id=331362e99eb14800b4d52472fd41a629&scope.project.id=f5b2e96e553d496d96e8f916fb522477&role.id=506b3cf33324478abe0f72a9d408aa54 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:04.737593 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49677e-ba56-47b0-b95a-1b70de38dc21 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:04.867466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756eee0-1f56-4d90-9454-e731b06091a0 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:04.908640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49677e-ba56-47b0-b95a-1b70de38dc21 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:04.967220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98713969-612a-4bc9-9862-ebe02454f9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:04.968229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98713969-612a-4bc9-9862-ebe02454f9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:04.968551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98713969-612a-4bc9-9862-ebe02454f9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:04.968842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98713969-612a-4bc9-9862-ebe02454f9a0 None admin] PATH_INFO: `/v3/projects/db9fa468925d45d9901c0bed2034e05f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:04.970192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98713969-612a-4bc9-9862-ebe02454f9a0 None admin] RBAC: Authorizing `identity:get_project(project_id=db9fa468925d45d9901c0bed2034e05f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:04.971688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98713969-612a-4bc9-9862-ebe02454f9a0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:04.982318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 92/178] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:04 2022] GET /identity/v3/projects/db9fa468925d45d9901c0bed2034e05f => generated 312 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:04.987959 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79fb6a-a115-488e-a2ee-e792418a9b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:04.988290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98713969-612a-4bc9-9862-ebe02454f9a0 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:04.989147 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79fb6a-a115-488e-a2ee-e792418a9b62 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:04.989651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79fb6a-a115-488e-a2ee-e792418a9b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:04.990151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79fb6a-a115-488e-a2ee-e792418a9b62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:04.991229 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79fb6a-a115-488e-a2ee-e792418a9b62 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=f5b2e96e553d496d96e8f916fb522477, user.id=3b98d845867d491c91aeca1e12cf3ace) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:04.991749 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79fb6a-a115-488e-a2ee-e792418a9b62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:04.993492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79fb6a-a115-488e-a2ee-e792418a9b62 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:05.003678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 87/179] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:04 2022] GET /identity/v3/role_assignments?user.id=3b98d845867d491c91aeca1e12cf3ace&scope.project.id=f5b2e96e553d496d96e8f916fb522477&role.id=584beecb022b422f820e77f7623ada5e => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:05.098038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:05.144729 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98713969-612a-4bc9-9862-ebe02454f9a0 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:05.184543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151c327-1ccd-4cb5-9749-ef0a6118c628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:05.185221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151c327-1ccd-4cb5-9749-ef0a6118c628 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:05.186058 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151c327-1ccd-4cb5-9749-ef0a6118c628 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:05.186366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151c327-1ccd-4cb5-9749-ef0a6118c628 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:05.188139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151c327-1ccd-4cb5-9749-ef0a6118c628 None admin] RBAC: Adding query filter params (role.id=e1a5192e7abc4046bc2c0673176ea1b6, scope.project.id=db9fa468925d45d9901c0bed2034e05f, user.id=331362e99eb14800b4d52472fd41a629) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:05.188139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151c327-1ccd-4cb5-9749-ef0a6118c628 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:05.189202 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151c327-1ccd-4cb5-9749-ef0a6118c628 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:05.196216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 93/180] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:04 2022] GET /identity/v3/role_assignments?user.id=331362e99eb14800b4d52472fd41a629&scope.project.id=db9fa468925d45d9901c0bed2034e05f&role.id=e1a5192e7abc4046bc2c0673176ea1b6 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:05.550985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc3b78-f34f-4b42-9ee9-c7480256263a None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:05.551338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc3b78-f34f-4b42-9ee9-c7480256263a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:05.551510 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc3b78-f34f-4b42-9ee9-c7480256263a None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:05.553020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 88/181] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:05.558918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c652f6b-f020-4a90-ade9-fbe48233e0eb None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:05.559571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c652f6b-f020-4a90-ade9-fbe48233e0eb None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:05.560139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c652f6b-f020-4a90-ade9-fbe48233e0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:05.579656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2c652f6b-f020-4a90-ade9-fbe48233e0eb None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:05.581195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2c652f6b-f020-4a90-ade9-fbe48233e0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:05.671903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 94/182] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:05.700568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc3b78-f34f-4b42-9ee9-c7480256263a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:05.704146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:05.840947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc3b78-f34f-4b42-9ee9-c7480256263a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:05.875893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67ac6f-ad1f-4478-8bf4-52a1d1636151 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:05.876716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67ac6f-ad1f-4478-8bf4-52a1d1636151 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:05.877641 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67ac6f-ad1f-4478-8bf4-52a1d1636151 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:05.879618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 95/183] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:03:05.884555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb58a93-d593-4940-80c2-94403d0002be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:05.885397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb58a93-d593-4940-80c2-94403d0002be None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:05.885682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb58a93-d593-4940-80c2-94403d0002be None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:05.886045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb58a93-d593-4940-80c2-94403d0002be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:05.886875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474d65e-6022-4679-af84-9a679df7da5f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:05.887367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474d65e-6022-4679-af84-9a679df7da5f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:05.887891 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474d65e-6022-4679-af84-9a679df7da5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:05.913337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb58a93-d593-4940-80c2-94403d0002be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-efb58a93-d593-4940-80c2-94403d0002be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:05.923625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:05.924583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:05.925401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:05.925401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:05.925401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:05.925401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:05.925401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:05.925401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:05.925401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:05.925401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:05.925401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:05.925401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:05.925401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:05.925401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:05.927258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4474d65e-6022-4679-af84-9a679df7da5f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:05.928084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 89/184] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:03:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:05.929026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4474d65e-6022-4679-af84-9a679df7da5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:05.936815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb58a93-d593-4940-80c2-94403d0002be None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:06.013173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:06.045708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:06.048595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 96/185] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:06.057814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732af2c-ab40-4aa4-9d8b-67937ddb8720 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:06.057814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732af2c-ab40-4aa4-9d8b-67937ddb8720 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:06.057814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732af2c-ab40-4aa4-9d8b-67937ddb8720 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:06.057814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 97/186] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:06.061111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d73e72-2ba4-43ee-9c07-5fcade960ccd None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:06.061401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d73e72-2ba4-43ee-9c07-5fcade960ccd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:06.061677 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d73e72-2ba4-43ee-9c07-5fcade960ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:06.081281 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-26d73e72-2ba4-43ee-9c07-5fcade960ccd None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:06.082503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-26d73e72-2ba4-43ee-9c07-5fcade960ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:06.100161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb58a93-d593-4940-80c2-94403d0002be None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:06.134805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7456a7-1cce-4d15-9f56-0c7b45ad1128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:06.135539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7456a7-1cce-4d15-9f56-0c7b45ad1128 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:06.136037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7456a7-1cce-4d15-9f56-0c7b45ad1128 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:06.136037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7456a7-1cce-4d15-9f56-0c7b45ad1128 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:06.136797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7456a7-1cce-4d15-9f56-0c7b45ad1128 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:06.137235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7456a7-1cce-4d15-9f56-0c7b45ad1128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:06.139193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7456a7-1cce-4d15-9f56-0c7b45ad1128 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:06.148070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 90/187] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:06.149094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7456a7-1cce-4d15-9f56-0c7b45ad1128 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:06.160797 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 98/188] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:06.162140 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca42a0a-c23a-48dd-8eb1-fed8477cb4eb None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:06.162423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca42a0a-c23a-48dd-8eb1-fed8477cb4eb None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:06.162694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca42a0a-c23a-48dd-8eb1-fed8477cb4eb None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:06.163949 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 99/189] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:06.164633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca42a0a-c23a-48dd-8eb1-fed8477cb4eb None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:06.256400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7456a7-1cce-4d15-9f56-0c7b45ad1128 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:06.276224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca42a0a-c23a-48dd-8eb1-fed8477cb4eb None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:06.280157 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:06.290566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:06.292009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8094a6-86dc-4032-b525-0020bf93146f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:06.292788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8094a6-86dc-4032-b525-0020bf93146f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:06.293066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8094a6-86dc-4032-b525-0020bf93146f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:06.293333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8094a6-86dc-4032-b525-0020bf93146f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:06.294720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8094a6-86dc-4032-b525-0020bf93146f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:06.296473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8094a6-86dc-4032-b525-0020bf93146f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:06.298106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 91/190] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:06.299253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739d989-4c5b-4176-b637-a7dfa378827a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:06.299533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739d989-4c5b-4176-b637-a7dfa378827a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:06.299801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739d989-4c5b-4176-b637-a7dfa378827a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:06.313407 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85cdcb-0806-4c3c-9508-c37054b1e6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:06.316762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85cdcb-0806-4c3c-9508-c37054b1e6af None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:06.316762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85cdcb-0806-4c3c-9508-c37054b1e6af None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:06.316762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85cdcb-0806-4c3c-9508-c37054b1e6af None admin] PATH_INFO: `/v3/users/2114c1df7f3f4c7bbeaf32b42b3bd74b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:06.318092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f739d989-4c5b-4176-b637-a7dfa378827a None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:06.319281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f739d989-4c5b-4176-b637-a7dfa378827a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:06.333522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85cdcb-0806-4c3c-9508-c37054b1e6af None admin] RBAC: Authorizing `identity:get_user(user_id=2114c1df7f3f4c7bbeaf32b42b3bd74b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:06.335015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85cdcb-0806-4c3c-9508-c37054b1e6af None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:06.336716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 100/191] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:06 2022] GET /identity/v3/users/2114c1df7f3f4c7bbeaf32b42b3bd74b => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:06.337488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85cdcb-0806-4c3c-9508-c37054b1e6af None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:06.386227 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 92/192] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:06.386958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739d989-4c5b-4176-b637-a7dfa378827a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:06.438188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85cdcb-0806-4c3c-9508-c37054b1e6af None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:06.471651 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2da0df-667a-4713-8e80-a8c0b79b0814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:06.472539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2da0df-667a-4713-8e80-a8c0b79b0814 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:06.472822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2da0df-667a-4713-8e80-a8c0b79b0814 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:06.473096 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2da0df-667a-4713-8e80-a8c0b79b0814 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:06.484118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2da0df-667a-4713-8e80-a8c0b79b0814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-4d2da0df-667a-4713-8e80-a8c0b79b0814 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:06.491761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:06.492619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:06.493393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 101/193] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:03:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:06.494688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2da0df-667a-4713-8e80-a8c0b79b0814 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:06.497224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739d989-4c5b-4176-b637-a7dfa378827a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:06.521904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:06.525134 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:06.534566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01db705-06d2-40ff-8556-55e33ba010ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:06.535508 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01db705-06d2-40ff-8556-55e33ba010ba None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:06.535788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01db705-06d2-40ff-8556-55e33ba010ba None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:06.536059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01db705-06d2-40ff-8556-55e33ba010ba None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:06.536765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01db705-06d2-40ff-8556-55e33ba010ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:06.538266 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01db705-06d2-40ff-8556-55e33ba010ba None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:06.558039 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 93/194] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:03:06 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:06.558856 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01db705-06d2-40ff-8556-55e33ba010ba None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:06.606235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2da0df-667a-4713-8e80-a8c0b79b0814 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:06.645473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f8225-6d7f-4779-810b-270f80a333ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:06.646166 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f8225-6d7f-4779-810b-270f80a333ce None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:06.646446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f8225-6d7f-4779-810b-270f80a333ce None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:06.646716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f8225-6d7f-4779-810b-270f80a333ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:06.681943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01db705-06d2-40ff-8556-55e33ba010ba None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:06.721322 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ff060-ace2-4532-9f3e-171c196f8b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:06.722515 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ff060-ace2-4532-9f3e-171c196f8b78 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:06.723030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ff060-ace2-4532-9f3e-171c196f8b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:06.723436 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ff060-ace2-4532-9f3e-171c196f8b78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:06.735272 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ff060-ace2-4532-9f3e-171c196f8b78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:06.736482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f8225-6d7f-4779-810b-270f80a333ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:06.738011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f8225-6d7f-4779-810b-270f80a333ce None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-fe4ff060-ace2-4532-9f3e-171c196f8b78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:06.744204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:06.745143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:06.746027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:06.746027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:06.746027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:06.746027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:06.746027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:06.746027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:06.746027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:06.746027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:06.746027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:06.746027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:06.746027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:06.746027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:06.748568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 94/195] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:06.749496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ff060-ace2-4532-9f3e-171c196f8b78 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:06.785191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:06.797302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:06.884786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ff060-ace2-4532-9f3e-171c196f8b78 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:06.905102 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 102/196] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:06.906027 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f8225-6d7f-4779-810b-270f80a333ce None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:06.937764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30c6b5-63e7-4004-95ae-f3045ac1fc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:06.938736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30c6b5-63e7-4004-95ae-f3045ac1fc06 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:06.939207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30c6b5-63e7-4004-95ae-f3045ac1fc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:06.939677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30c6b5-63e7-4004-95ae-f3045ac1fc06 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:06.941059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30c6b5-63e7-4004-95ae-f3045ac1fc06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:06.941496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30c6b5-63e7-4004-95ae-f3045ac1fc06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:06.943779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30c6b5-63e7-4004-95ae-f3045ac1fc06 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:06.961204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 95/197] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:06.967591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30c6b5-63e7-4004-95ae-f3045ac1fc06 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:07.067942 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f8225-6d7f-4779-810b-270f80a333ce None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:07.106250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:07.120944 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7da955-986a-4994-bb53-6562523b6bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:07.121571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7da955-986a-4994-bb53-6562523b6bbc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:07.121867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7da955-986a-4994-bb53-6562523b6bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:07.122147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7da955-986a-4994-bb53-6562523b6bbc None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:07.123535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7da955-986a-4994-bb53-6562523b6bbc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:07.127028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7da955-986a-4994-bb53-6562523b6bbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:07.127028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7da955-986a-4994-bb53-6562523b6bbc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:07.129991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:07.133470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30c6b5-63e7-4004-95ae-f3045ac1fc06 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:07.143432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 103/198] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:07.144711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7da955-986a-4994-bb53-6562523b6bbc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:07.183766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639992f-3c80-496f-9c7b-0d559cb488aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:07.184965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639992f-3c80-496f-9c7b-0d559cb488aa None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:07.186164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639992f-3c80-496f-9c7b-0d559cb488aa None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:07.186793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639992f-3c80-496f-9c7b-0d559cb488aa None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:07.187939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639992f-3c80-496f-9c7b-0d559cb488aa None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:07.190116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639992f-3c80-496f-9c7b-0d559cb488aa None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:07.192566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 96/199] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:06 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:07.193291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639992f-3c80-496f-9c7b-0d559cb488aa None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:07.291776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7da955-986a-4994-bb53-6562523b6bbc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:07.329838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639992f-3c80-496f-9c7b-0d559cb488aa None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:07.338467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc9c24-3ead-469d-b7a2-cba0ceb7246b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:07.339344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc9c24-3ead-469d-b7a2-cba0ceb7246b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:07.339622 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc9c24-3ead-469d-b7a2-cba0ceb7246b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:07.339898 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc9c24-3ead-469d-b7a2-cba0ceb7246b None admin] PATH_INFO: `/v3/users/2114c1df7f3f4c7bbeaf32b42b3bd74b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:07.341289 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc9c24-3ead-469d-b7a2-cba0ceb7246b None admin] RBAC: Authorizing `identity:get_user(user_id=2114c1df7f3f4c7bbeaf32b42b3bd74b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:07.342998 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc9c24-3ead-469d-b7a2-cba0ceb7246b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:07.344823 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 104/200] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:07 2022] GET /identity/v3/users/2114c1df7f3f4c7bbeaf32b42b3bd74b => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:07.345759 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc9c24-3ead-469d-b7a2-cba0ceb7246b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:07.377516 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9184cfd-9146-4a7b-ac87-5969476afd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:07.379524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9184cfd-9146-4a7b-ac87-5969476afd10 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:07.380136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9184cfd-9146-4a7b-ac87-5969476afd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:07.380873 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9184cfd-9146-4a7b-ac87-5969476afd10 None admin] PATH_INFO: `/v3/users/127a73b4627c49c6902e1cd23d25169e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:07.383088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9184cfd-9146-4a7b-ac87-5969476afd10 None admin] RBAC: Authorizing `identity:get_user(user_id=127a73b4627c49c6902e1cd23d25169e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:07.385084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9184cfd-9146-4a7b-ac87-5969476afd10 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:07.387932 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 97/201] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:07 2022] GET /identity/v3/users/127a73b4627c49c6902e1cd23d25169e => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:07.388991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9184cfd-9146-4a7b-ac87-5969476afd10 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:07.415696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:07.447246 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:07.471914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc9c24-3ead-469d-b7a2-cba0ceb7246b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:07.516998 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c094be-4146-479a-8bf3-8b3fdff7c783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:07.517883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c094be-4146-479a-8bf3-8b3fdff7c783 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:07.518170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c094be-4146-479a-8bf3-8b3fdff7c783 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:07.518447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c094be-4146-479a-8bf3-8b3fdff7c783 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:07.519738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c094be-4146-479a-8bf3-8b3fdff7c783 None admin] RBAC: Adding query filter params (role.id=506b3cf33324478abe0f72a9d408aa54, scope.project.id=fea9cdff50914413b637910a718a6cd5, user.id=2114c1df7f3f4c7bbeaf32b42b3bd74b) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:07.520059 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c094be-4146-479a-8bf3-8b3fdff7c783 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:07.521811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c094be-4146-479a-8bf3-8b3fdff7c783 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:07.530333 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 105/202] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:07 2022] GET /identity/v3/role_assignments?user.id=2114c1df7f3f4c7bbeaf32b42b3bd74b&scope.project.id=fea9cdff50914413b637910a718a6cd5&role.id=506b3cf33324478abe0f72a9d408aa54 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:07.534974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c094be-4146-479a-8bf3-8b3fdff7c783 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:07.561492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9184cfd-9146-4a7b-ac87-5969476afd10 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:07.672239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16268627-53b9-4706-8141-a615a54bb2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:07.673299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16268627-53b9-4706-8141-a615a54bb2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:07.673810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16268627-53b9-4706-8141-a615a54bb2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:07.675071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16268627-53b9-4706-8141-a615a54bb2b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:07.734416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c094be-4146-479a-8bf3-8b3fdff7c783 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:07.770368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:07.786195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfac14-598b-4bfa-8234-9bc16a22fc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:07.786859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfac14-598b-4bfa-8234-9bc16a22fc9a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:07.787143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfac14-598b-4bfa-8234-9bc16a22fc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:07.787419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfac14-598b-4bfa-8234-9bc16a22fc9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:07.817370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16268627-53b9-4706-8141-a615a54bb2b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:07.819041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16268627-53b9-4706-8141-a615a54bb2b4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:07.891910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfac14-598b-4bfa-8234-9bc16a22fc9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:07.893423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfac14-598b-4bfa-8234-9bc16a22fc9a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:07.935595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:08.049257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 98/203] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:08.052262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 106/204] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:08.058833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfac14-598b-4bfa-8234-9bc16a22fc9a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:08.059000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16268627-53b9-4706-8141-a615a54bb2b4 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:08.110156 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:08.212038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfac14-598b-4bfa-8234-9bc16a22fc9a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:08.252649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440576f0-1021-4c50-9a0b-aecc206f6004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:08.253522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440576f0-1021-4c50-9a0b-aecc206f6004 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:08.253836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440576f0-1021-4c50-9a0b-aecc206f6004 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:08.254119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440576f0-1021-4c50-9a0b-aecc206f6004 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:08.255349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440576f0-1021-4c50-9a0b-aecc206f6004 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:08.256827 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440576f0-1021-4c50-9a0b-aecc206f6004 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:08.258597 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 107/205] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:08 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:08.268678 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440576f0-1021-4c50-9a0b-aecc206f6004 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:08.302085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16268627-53b9-4706-8141-a615a54bb2b4 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:08.344811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fbe94-635e-4997-9239-eed0b7b48bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:08.346686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fbe94-635e-4997-9239-eed0b7b48bef None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:08.347183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fbe94-635e-4997-9239-eed0b7b48bef None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:08.347625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fbe94-635e-4997-9239-eed0b7b48bef None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:08.353893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fbe94-635e-4997-9239-eed0b7b48bef None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:08.353893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fbe94-635e-4997-9239-eed0b7b48bef None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:08.354303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 99/206] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:08 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:08.365329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fbe94-635e-4997-9239-eed0b7b48bef None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:08.370067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:08.474978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:08.492083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440576f0-1021-4c50-9a0b-aecc206f6004 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:08.556360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21845a0c-96a4-4a40-aee4-6cec0a29c547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:08.556930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21845a0c-96a4-4a40-aee4-6cec0a29c547 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:08.557211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21845a0c-96a4-4a40-aee4-6cec0a29c547 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:08.557481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21845a0c-96a4-4a40-aee4-6cec0a29c547 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:08.558346 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21845a0c-96a4-4a40-aee4-6cec0a29c547 None admin] RBAC: Adding query filter params (role.id=45c97834bc614c5ca5122d691ce93316, scope.project.id=f5b2e96e553d496d96e8f916fb522477, user.id=127a73b4627c49c6902e1cd23d25169e) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:08.558662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21845a0c-96a4-4a40-aee4-6cec0a29c547 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:08.560273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21845a0c-96a4-4a40-aee4-6cec0a29c547 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:08.564636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fbe94-635e-4997-9239-eed0b7b48bef None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:08.575382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 108/207] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:08 2022] GET /identity/v3/role_assignments?user.id=127a73b4627c49c6902e1cd23d25169e&scope.project.id=f5b2e96e553d496d96e8f916fb522477&role.id=45c97834bc614c5ca5122d691ce93316 => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:08.624846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492a67b-b4b9-46ad-83b6-30e11bd90961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:08.625519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492a67b-b4b9-46ad-83b6-30e11bd90961 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:08.625852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492a67b-b4b9-46ad-83b6-30e11bd90961 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:08.626262 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492a67b-b4b9-46ad-83b6-30e11bd90961 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:08.627186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492a67b-b4b9-46ad-83b6-30e11bd90961 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=fea9cdff50914413b637910a718a6cd5, user.id=2114c1df7f3f4c7bbeaf32b42b3bd74b) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:08.627564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492a67b-b4b9-46ad-83b6-30e11bd90961 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:08.629108 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492a67b-b4b9-46ad-83b6-30e11bd90961 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:08.640872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 100/208] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:08 2022] GET /identity/v3/role_assignments?user.id=2114c1df7f3f4c7bbeaf32b42b3bd74b&scope.project.id=fea9cdff50914413b637910a718a6cd5&role.id=584beecb022b422f820e77f7623ada5e => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:09.166287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5109f9f-e72f-4311-a86e-45b89a67e993 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:09.166795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5109f9f-e72f-4311-a86e-45b89a67e993 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:09.167108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5109f9f-e72f-4311-a86e-45b89a67e993 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:09.168712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 109/209] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:09.174998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65adabe-aaed-4c5b-b81b-0fae39a50a55 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:09.175858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65adabe-aaed-4c5b-b81b-0fae39a50a55 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:09.176337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65adabe-aaed-4c5b-b81b-0fae39a50a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:09.195174 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d65adabe-aaed-4c5b-b81b-0fae39a50a55 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:09.196704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d65adabe-aaed-4c5b-b81b-0fae39a50a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:09.286075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:09.289842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 101/210] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:09.309595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5109f9f-e72f-4311-a86e-45b89a67e993 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:09.421751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5109f9f-e72f-4311-a86e-45b89a67e993 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:09.434315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69329a4-396d-49a6-94c8-9d798bc31a2c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:09.434912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69329a4-396d-49a6-94c8-9d798bc31a2c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:09.435931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69329a4-396d-49a6-94c8-9d798bc31a2c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:09.437661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 102/211] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:03:09.443214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062ec51-c72f-48a5-aa58-0a509b103b1f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:09.443786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062ec51-c72f-48a5-aa58-0a509b103b1f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:09.444285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062ec51-c72f-48a5-aa58-0a509b103b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:09.457353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098ab29-a6c2-45b7-88b0-b1eda7a861e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:09.458171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098ab29-a6c2-45b7-88b0-b1eda7a861e7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:09.458451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098ab29-a6c2-45b7-88b0-b1eda7a861e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:09.458780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098ab29-a6c2-45b7-88b0-b1eda7a861e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:09.463702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5062ec51-c72f-48a5-aa58-0a509b103b1f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:09.465074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5062ec51-c72f-48a5-aa58-0a509b103b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:09.470159 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098ab29-a6c2-45b7-88b0-b1eda7a861e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f098ab29-a6c2-45b7-88b0-b1eda7a861e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:09.478033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:09.478940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:09.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 110/212] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:09.485904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098ab29-a6c2-45b7-88b0-b1eda7a861e7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:09.509572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:09.574024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 103/213] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:09.600340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062ec51-c72f-48a5-aa58-0a509b103b1f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:09.612809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098ab29-a6c2-45b7-88b0-b1eda7a861e7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:09.663699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284aab86-7b7b-41f0-adda-fc25dda88062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:09.664777 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284aab86-7b7b-41f0-adda-fc25dda88062 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:09.665477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284aab86-7b7b-41f0-adda-fc25dda88062 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:09.666394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284aab86-7b7b-41f0-adda-fc25dda88062 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:09.667601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284aab86-7b7b-41f0-adda-fc25dda88062 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:09.668089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284aab86-7b7b-41f0-adda-fc25dda88062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:09.670430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284aab86-7b7b-41f0-adda-fc25dda88062 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:09.682325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 111/214] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:09.690406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284aab86-7b7b-41f0-adda-fc25dda88062 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:09.717391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062ec51-c72f-48a5-aa58-0a509b103b1f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:09.745559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:09.760357 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf06bd-320a-43bf-a033-84b29aec4732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:09.761141 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf06bd-320a-43bf-a033-84b29aec4732 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:09.761415 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf06bd-320a-43bf-a033-84b29aec4732 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:09.761681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf06bd-320a-43bf-a033-84b29aec4732 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:09.764608 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf06bd-320a-43bf-a033-84b29aec4732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:09.766149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf06bd-320a-43bf-a033-84b29aec4732 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:09.772812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 104/215] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:09.774016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf06bd-320a-43bf-a033-84b29aec4732 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:09.804067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:09.835747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284aab86-7b7b-41f0-adda-fc25dda88062 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:09.884117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000e205-9d14-4ae2-8d29-5ad1a9da3156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:09.885913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000e205-9d14-4ae2-8d29-5ad1a9da3156 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:09.886627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000e205-9d14-4ae2-8d29-5ad1a9da3156 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:09.887064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000e205-9d14-4ae2-8d29-5ad1a9da3156 None admin] PATH_INFO: `/v3/users/13fb018ff58842608881eb26b024a95a` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:09.893169 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf06bd-320a-43bf-a033-84b29aec4732 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:09.907565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000e205-9d14-4ae2-8d29-5ad1a9da3156 None admin] RBAC: Authorizing `identity:get_user(user_id=13fb018ff58842608881eb26b024a95a)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:09.909638 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000e205-9d14-4ae2-8d29-5ad1a9da3156 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:09.912846 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 112/216] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:09 2022] GET /identity/v3/users/13fb018ff58842608881eb26b024a95a => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:09.918960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000e205-9d14-4ae2-8d29-5ad1a9da3156 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:09.952563 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc3fde-f68f-4214-b824-90df2fc55619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:09.953153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc3fde-f68f-4214-b824-90df2fc55619 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:09.953431 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc3fde-f68f-4214-b824-90df2fc55619 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:09.953699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc3fde-f68f-4214-b824-90df2fc55619 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:09.955109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc3fde-f68f-4214-b824-90df2fc55619 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:09.956579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc3fde-f68f-4214-b824-90df2fc55619 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:09.993126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 105/217] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:03:09 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:10.048591 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000e205-9d14-4ae2-8d29-5ad1a9da3156 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:10.088148 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e845d58-a56a-4433-81be-14de83006db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:10.089205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e845d58-a56a-4433-81be-14de83006db4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:10.089691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e845d58-a56a-4433-81be-14de83006db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:10.090211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e845d58-a56a-4433-81be-14de83006db4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:10.095472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:10.113372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50902ab0-ca8c-464d-8b03-e0fb4dc49399 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:10.113720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50902ab0-ca8c-464d-8b03-e0fb4dc49399 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:10.113978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50902ab0-ca8c-464d-8b03-e0fb4dc49399 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:10.115447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 106/218] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:10.121084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330a503-f6f0-44a1-a3f8-7f86ff4fe6fa None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:10.121374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330a503-f6f0-44a1-a3f8-7f86ff4fe6fa None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:10.121651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330a503-f6f0-44a1-a3f8-7f86ff4fe6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:10.143318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b330a503-f6f0-44a1-a3f8-7f86ff4fe6fa None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:10.144566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b330a503-f6f0-44a1-a3f8-7f86ff4fe6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:10.216866 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e845d58-a56a-4433-81be-14de83006db4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:10.217775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:10.218369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e845d58-a56a-4433-81be-14de83006db4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:10.250538 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 107/219] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:10.274828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330a503-f6f0-44a1-a3f8-7f86ff4fe6fa None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:10.359100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 113/220] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:10.365375 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e845d58-a56a-4433-81be-14de83006db4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:10.403471 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330a503-f6f0-44a1-a3f8-7f86ff4fe6fa None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:10.418119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:10.447527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13d580-d6be-440c-a8a4-ed7b0d36e930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:10.448502 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13d580-d6be-440c-a8a4-ed7b0d36e930 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:10.448785 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13d580-d6be-440c-a8a4-ed7b0d36e930 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:10.449375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13d580-d6be-440c-a8a4-ed7b0d36e930 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:10.469578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f13d580-d6be-440c-a8a4-ed7b0d36e930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9f13d580-d6be-440c-a8a4-ed7b0d36e930 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:10.481208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:10.482891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:10.483907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 108/221] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:03:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:10.490627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13d580-d6be-440c-a8a4-ed7b0d36e930 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:10.510784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e845d58-a56a-4433-81be-14de83006db4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:10.549126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:10.577565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6ab45-8e7d-41a8-bb91-f80d1833e754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:10.578456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6ab45-8e7d-41a8-bb91-f80d1833e754 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:10.578733 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6ab45-8e7d-41a8-bb91-f80d1833e754 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:10.579001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6ab45-8e7d-41a8-bb91-f80d1833e754 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:10.580457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6ab45-8e7d-41a8-bb91-f80d1833e754 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:10.582600 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6ab45-8e7d-41a8-bb91-f80d1833e754 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:10.585157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 114/222] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:10 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:10.591073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6ab45-8e7d-41a8-bb91-f80d1833e754 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:10.657492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13d580-d6be-440c-a8a4-ed7b0d36e930 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:10.725363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391800c-6769-47ec-a156-c2817be66b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:10.726397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391800c-6769-47ec-a156-c2817be66b29 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:10.726882 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391800c-6769-47ec-a156-c2817be66b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:10.727352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391800c-6769-47ec-a156-c2817be66b29 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:10.728462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391800c-6769-47ec-a156-c2817be66b29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:10.728969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391800c-6769-47ec-a156-c2817be66b29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:10.730609 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391800c-6769-47ec-a156-c2817be66b29 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:10.740136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 109/223] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:10.746755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391800c-6769-47ec-a156-c2817be66b29 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:10.801503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6ab45-8e7d-41a8-bb91-f80d1833e754 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:10.809603 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:10.884447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391800c-6769-47ec-a156-c2817be66b29 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:10.886942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:10.890656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e6f3b-5c33-4da9-b073-69eec1fe8e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:10.891337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e6f3b-5c33-4da9-b073-69eec1fe8e18 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:10.891621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e6f3b-5c33-4da9-b073-69eec1fe8e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:10.891896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e6f3b-5c33-4da9-b073-69eec1fe8e18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:10.892734 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e6f3b-5c33-4da9-b073-69eec1fe8e18 None admin] RBAC: Adding query filter params (role.id=e1a5192e7abc4046bc2c0673176ea1b6, scope.project.id=fea9cdff50914413b637910a718a6cd5, user.id=13fb018ff58842608881eb26b024a95a) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:10.893048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e6f3b-5c33-4da9-b073-69eec1fe8e18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:10.894533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e6f3b-5c33-4da9-b073-69eec1fe8e18 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:10.911783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 115/224] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:10 2022] GET /identity/v3/role_assignments?user.id=13fb018ff58842608881eb26b024a95a&scope.project.id=fea9cdff50914413b637910a718a6cd5&role.id=e1a5192e7abc4046bc2c0673176ea1b6 => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:10.928495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a653d-e85c-4cdd-a7b1-b7d1f56d2c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:10.929779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a653d-e85c-4cdd-a7b1-b7d1f56d2c70 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:10.930264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a653d-e85c-4cdd-a7b1-b7d1f56d2c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:10.930768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a653d-e85c-4cdd-a7b1-b7d1f56d2c70 None admin] PATH_INFO: `/v3/users/2114c1df7f3f4c7bbeaf32b42b3bd74b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:10.932173 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a653d-e85c-4cdd-a7b1-b7d1f56d2c70 None admin] RBAC: Authorizing `identity:get_user(user_id=2114c1df7f3f4c7bbeaf32b42b3bd74b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:10.933882 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a653d-e85c-4cdd-a7b1-b7d1f56d2c70 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:10.936373 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 110/225] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:10 2022] GET /identity/v3/users/2114c1df7f3f4c7bbeaf32b42b3bd74b => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:10.942320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e6f3b-5c33-4da9-b073-69eec1fe8e18 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:11.059615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc07e0f-fec3-495d-9b1c-96a4d0149381 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:11.060278 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc07e0f-fec3-495d-9b1c-96a4d0149381 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:11.060778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc07e0f-fec3-495d-9b1c-96a4d0149381 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:11.062833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 111/226] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:11.069042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff8ae0-6b4a-440a-b4db-6d70b1215ead None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:11.069652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff8ae0-6b4a-440a-b4db-6d70b1215ead None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:11.070317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff8ae0-6b4a-440a-b4db-6d70b1215ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:11.071738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e6f3b-5c33-4da9-b073-69eec1fe8e18 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:11.111122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-0cff8ae0-6b4a-440a-b4db-6d70b1215ead None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:11.112690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0cff8ae0-6b4a-440a-b4db-6d70b1215ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:11.127863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ad5d9-6fa7-4c66-8d13-5adfec7b6db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:11.129118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ad5d9-6fa7-4c66-8d13-5adfec7b6db9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:11.129601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ad5d9-6fa7-4c66-8d13-5adfec7b6db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:11.130095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ad5d9-6fa7-4c66-8d13-5adfec7b6db9 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:11.133398 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ad5d9-6fa7-4c66-8d13-5adfec7b6db9 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:11.134901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ad5d9-6fa7-4c66-8d13-5adfec7b6db9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:11.136784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 116/227] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:10 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:11.143913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ad5d9-6fa7-4c66-8d13-5adfec7b6db9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:11.198249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 112/228] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:11.210181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:11.223049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff8ae0-6b4a-440a-b4db-6d70b1215ead None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:11.339520 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff8ae0-6b4a-440a-b4db-6d70b1215ead None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:11.341660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ad5d9-6fa7-4c66-8d13-5adfec7b6db9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:11.342107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:11.375660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6df5b-c906-422d-bb62-4d1302cf6842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:11.376454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6df5b-c906-422d-bb62-4d1302cf6842 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:11.376731 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6df5b-c906-422d-bb62-4d1302cf6842 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:11.377000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6df5b-c906-422d-bb62-4d1302cf6842 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:11.387514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6df5b-c906-422d-bb62-4d1302cf6842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c2f6df5b-c906-422d-bb62-4d1302cf6842 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:11.403580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:11.404500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:11.405274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 113/229] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:03:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:11.412578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cc2be-b134-451f-826b-58e44d2eb3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:11.413469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cc2be-b134-451f-826b-58e44d2eb3c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:11.413765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cc2be-b134-451f-826b-58e44d2eb3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:11.414061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cc2be-b134-451f-826b-58e44d2eb3c0 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5/users/2114c1df7f3f4c7bbeaf32b42b3bd74b/roles/506b3cf33324478abe0f72a9d408aa54` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:11.416509 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6df5b-c906-422d-bb62-4d1302cf6842 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:11.424556 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cc2be-b134-451f-826b-58e44d2eb3c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=fea9cdff50914413b637910a718a6cd5, user_id=2114c1df7f3f4c7bbeaf32b42b3bd74b, role_id=506b3cf33324478abe0f72a9d408aa54)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:11.426192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cc2be-b134-451f-826b-58e44d2eb3c0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:11.473213 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 117/230] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:03:11 2022] PUT /identity/v3/projects/fea9cdff50914413b637910a718a6cd5/users/2114c1df7f3f4c7bbeaf32b42b3bd74b/roles/506b3cf33324478abe0f72a9d408aa54 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:03:11.543494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6df5b-c906-422d-bb62-4d1302cf6842 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:11.580535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bffa98-59a0-4a2b-b972-a50c3b279e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:11.581134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bffa98-59a0-4a2b-b972-a50c3b279e43 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:11.581463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bffa98-59a0-4a2b-b972-a50c3b279e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:11.581754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bffa98-59a0-4a2b-b972-a50c3b279e43 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:11.582527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bffa98-59a0-4a2b-b972-a50c3b279e43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:11.582834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bffa98-59a0-4a2b-b972-a50c3b279e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:11.584255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bffa98-59a0-4a2b-b972-a50c3b279e43 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:11.594262 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 114/231] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:11.602945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cc2be-b134-451f-826b-58e44d2eb3c0 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:11.733884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:11.740163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cc2be-b134-451f-826b-58e44d2eb3c0 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:11.790063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5bdc2-c808-462e-bb86-c3cde6675e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:11.791019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5bdc2-c808-462e-bb86-c3cde6675e01 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:11.791301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5bdc2-c808-462e-bb86-c3cde6675e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:11.791571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5bdc2-c808-462e-bb86-c3cde6675e01 None admin] PATH_INFO: `/v3/users/2114c1df7f3f4c7bbeaf32b42b3bd74b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:11.792835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5bdc2-c808-462e-bb86-c3cde6675e01 None admin] RBAC: Authorizing `identity:get_user(user_id=2114c1df7f3f4c7bbeaf32b42b3bd74b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:11.794952 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5bdc2-c808-462e-bb86-c3cde6675e01 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:11.796960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 118/232] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:11 2022] GET /identity/v3/users/2114c1df7f3f4c7bbeaf32b42b3bd74b => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:11.802591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bffa98-59a0-4a2b-b972-a50c3b279e43 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:11.840217 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:11.931111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bffa98-59a0-4a2b-b972-a50c3b279e43 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:11.967103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0938e-30db-4722-833e-8b46b8ae9b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:11.968019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0938e-30db-4722-833e-8b46b8ae9b14 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:11.968296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0938e-30db-4722-833e-8b46b8ae9b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:11.968608 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0938e-30db-4722-833e-8b46b8ae9b14 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:11.969895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0938e-30db-4722-833e-8b46b8ae9b14 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:11.971364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0938e-30db-4722-833e-8b46b8ae9b14 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:11.973061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 115/233] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:11 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:11.978822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5bdc2-c808-462e-bb86-c3cde6675e01 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:12.094627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5bdc2-c808-462e-bb86-c3cde6675e01 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:12.141474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15b45c-49ab-4bdf-989b-a9f9109cc8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:12.142401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15b45c-49ab-4bdf-989b-a9f9109cc8b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:12.142684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15b45c-49ab-4bdf-989b-a9f9109cc8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:12.142961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15b45c-49ab-4bdf-989b-a9f9109cc8b9 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5/users/2114c1df7f3f4c7bbeaf32b42b3bd74b/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:12.151061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15b45c-49ab-4bdf-989b-a9f9109cc8b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=fea9cdff50914413b637910a718a6cd5, user_id=2114c1df7f3f4c7bbeaf32b42b3bd74b, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:12.152541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15b45c-49ab-4bdf-989b-a9f9109cc8b9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:12.180371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 119/234] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:03:11 2022] PUT /identity/v3/projects/fea9cdff50914413b637910a718a6cd5/users/2114c1df7f3f4c7bbeaf32b42b3bd74b/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:12.859875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c378ac9-ffff-4e16-995e-92d274668ba4 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:12.861143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c378ac9-ffff-4e16-995e-92d274668ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:12.861599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c378ac9-ffff-4e16-995e-92d274668ba4 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:12.863568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 116/235] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:12.875747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057d7f4-ffd1-4f9c-a276-b55d97f216b4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:12.876579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057d7f4-ffd1-4f9c-a276-b55d97f216b4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:12.877001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057d7f4-ffd1-4f9c-a276-b55d97f216b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:12.903492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a057d7f4-ffd1-4f9c-a276-b55d97f216b4 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:12.905591 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a057d7f4-ffd1-4f9c-a276-b55d97f216b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:12.930687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:12.996955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 120/236] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:13.046198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c378ac9-ffff-4e16-995e-92d274668ba4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:13.147456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cfce9-b020-4d79-ac7b-0430d4b24e24 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:13.147964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cfce9-b020-4d79-ac7b-0430d4b24e24 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:13.148380 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cfce9-b020-4d79-ac7b-0430d4b24e24 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:13.150534 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 121/237] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:03:13.157571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba993b-6102-4428-baa9-cb11994ffa04 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:13.158213 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba993b-6102-4428-baa9-cb11994ffa04 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:13.158954 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba993b-6102-4428-baa9-cb11994ffa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:13.181797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:13.192657 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c378ac9-ffff-4e16-995e-92d274668ba4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:13.193931 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-11ba993b-6102-4428-baa9-cb11994ffa04 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:13.195317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-11ba993b-6102-4428-baa9-cb11994ffa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:13.236329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4309759-def5-41fe-bab1-28fa73dfdbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:13.237930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4309759-def5-41fe-bab1-28fa73dfdbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:13.238435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4309759-def5-41fe-bab1-28fa73dfdbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:13.238998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4309759-def5-41fe-bab1-28fa73dfdbd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:13.241049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4309759-def5-41fe-bab1-28fa73dfdbd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:13.243669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4309759-def5-41fe-bab1-28fa73dfdbd3 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:13.246915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 117/238] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:13.259179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4309759-def5-41fe-bab1-28fa73dfdbd3 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:13.335226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 122/239] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:13.371025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba993b-6102-4428-baa9-cb11994ffa04 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:13.412454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4309759-def5-41fe-bab1-28fa73dfdbd3 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:13.457053 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd14fa4-66e7-4a1f-b88f-d56788ca5c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:13.457988 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd14fa4-66e7-4a1f-b88f-d56788ca5c9f None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:13.458439 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd14fa4-66e7-4a1f-b88f-d56788ca5c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:13.458872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd14fa4-66e7-4a1f-b88f-d56788ca5c9f None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:13.459714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd14fa4-66e7-4a1f-b88f-d56788ca5c9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:13.461314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd14fa4-66e7-4a1f-b88f-d56788ca5c9f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:13.499294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 118/240] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:03:13 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:13.548133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:13.574449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba993b-6102-4428-baa9-cb11994ffa04 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:13.619589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346f3e0-bf73-41db-b3b5-9e14c53df2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:13.620387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346f3e0-bf73-41db-b3b5-9e14c53df2ef None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:13.620665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346f3e0-bf73-41db-b3b5-9e14c53df2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:13.620933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346f3e0-bf73-41db-b3b5-9e14c53df2ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:13.638820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346f3e0-bf73-41db-b3b5-9e14c53df2ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b346f3e0-bf73-41db-b3b5-9e14c53df2ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:13.646642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:13.647500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:13.648292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 123/241] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:13.654338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd14fa4-66e7-4a1f-b88f-d56788ca5c9f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:13.688623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:13.721685 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25429-ee22-4cd8-8c94-1fc7cdf08692 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:13.722204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25429-ee22-4cd8-8c94-1fc7cdf08692 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:13.722679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25429-ee22-4cd8-8c94-1fc7cdf08692 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:13.725306 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 124/242] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:13.732441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f8275-4bc9-42df-b542-3174bba11925 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:13.732761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f8275-4bc9-42df-b542-3174bba11925 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:13.733043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f8275-4bc9-42df-b542-3174bba11925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:13.770529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-db4f8275-4bc9-42df-b542-3174bba11925 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:13.772719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-db4f8275-4bc9-42df-b542-3174bba11925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:13.829901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd14fa4-66e7-4a1f-b88f-d56788ca5c9f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:13.867490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59869b42-bd1c-428a-b9e1-edb99408c6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:13.868216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59869b42-bd1c-428a-b9e1-edb99408c6ca None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:13.868445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59869b42-bd1c-428a-b9e1-edb99408c6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:13.868718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59869b42-bd1c-428a-b9e1-edb99408c6ca None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:13.869480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59869b42-bd1c-428a-b9e1-edb99408c6ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:13.869806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59869b42-bd1c-428a-b9e1-edb99408c6ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:13.873417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59869b42-bd1c-428a-b9e1-edb99408c6ca None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:13.881059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 119/243] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:13.886489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59869b42-bd1c-428a-b9e1-edb99408c6ca None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:13.894095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 125/244] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:13.917974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f8275-4bc9-42df-b542-3174bba11925 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:13.989484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:14.018336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:14.025540 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59869b42-bd1c-428a-b9e1-edb99408c6ca None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:14.049689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f8275-4bc9-42df-b542-3174bba11925 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:14.071355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a30d5-bf01-418e-be4a-bc1afe82255d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:14.072232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a30d5-bf01-418e-be4a-bc1afe82255d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:14.072507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a30d5-bf01-418e-be4a-bc1afe82255d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:14.072774 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a30d5-bf01-418e-be4a-bc1afe82255d None admin] PATH_INFO: `/v3/users/579c35a891a945078195650268cbf6e2` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:14.090119 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a30d5-bf01-418e-be4a-bc1afe82255d None admin] RBAC: Authorizing `identity:get_user(user_id=579c35a891a945078195650268cbf6e2)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:14.092185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a30d5-bf01-418e-be4a-bc1afe82255d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:14.094115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 120/245] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:13 2022] GET /identity/v3/users/579c35a891a945078195650268cbf6e2 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:14.100855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a30d5-bf01-418e-be4a-bc1afe82255d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:14.108311 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b92b1-bcc5-434b-b2b6-7b47ddfc473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:14.109117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b92b1-bcc5-434b-b2b6-7b47ddfc473c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:14.109398 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b92b1-bcc5-434b-b2b6-7b47ddfc473c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:14.109669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b92b1-bcc5-434b-b2b6-7b47ddfc473c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:14.130543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b92b1-bcc5-434b-b2b6-7b47ddfc473c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-9b5b92b1-bcc5-434b-b2b6-7b47ddfc473c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:14.141356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:14.142282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:14.143050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 126/246] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:14.148626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b92b1-bcc5-434b-b2b6-7b47ddfc473c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:14.238586 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a30d5-bf01-418e-be4a-bc1afe82255d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:14.285929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b92b1-bcc5-434b-b2b6-7b47ddfc473c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:14.289010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016170f0-15b5-4289-9b1c-96ec915f5cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:14.289817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016170f0-15b5-4289-9b1c-96ec915f5cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:14.290218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016170f0-15b5-4289-9b1c-96ec915f5cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:14.290594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016170f0-15b5-4289-9b1c-96ec915f5cf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:14.295874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:14.339091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dec07-4582-4c30-94ca-8ef2c33dada1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:14.339776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dec07-4582-4c30-94ca-8ef2c33dada1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:14.340062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dec07-4582-4c30-94ca-8ef2c33dada1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:14.340345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dec07-4582-4c30-94ca-8ef2c33dada1 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:14.341898 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dec07-4582-4c30-94ca-8ef2c33dada1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:14.343568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dec07-4582-4c30-94ca-8ef2c33dada1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:14.345184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dec07-4582-4c30-94ca-8ef2c33dada1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:14.348537 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:14.358393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 127/247] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:14.364593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dec07-4582-4c30-94ca-8ef2c33dada1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:14.379275 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016170f0-15b5-4289-9b1c-96ec915f5cf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:14.381080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016170f0-15b5-4289-9b1c-96ec915f5cf2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:14.480141 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dec07-4582-4c30-94ca-8ef2c33dada1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:14.485525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 121/248] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:14.491271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016170f0-15b5-4289-9b1c-96ec915f5cf2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:14.524530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd0b6c-f998-472a-984b-5ebf787bc9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:14.525415 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd0b6c-f998-472a-984b-5ebf787bc9fb None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:14.525704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd0b6c-f998-472a-984b-5ebf787bc9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:14.526006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd0b6c-f998-472a-984b-5ebf787bc9fb None admin] PATH_INFO: `/v3/users/13fb018ff58842608881eb26b024a95a` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:14.528375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:14.529834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd0b6c-f998-472a-984b-5ebf787bc9fb None admin] RBAC: Authorizing `identity:get_user(user_id=13fb018ff58842608881eb26b024a95a)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:14.531543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd0b6c-f998-472a-984b-5ebf787bc9fb None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:14.533565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 128/249] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:14 2022] GET /identity/v3/users/13fb018ff58842608881eb26b024a95a => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:14.539286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd0b6c-f998-472a-984b-5ebf787bc9fb None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:14.624145 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016170f0-15b5-4289-9b1c-96ec915f5cf2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:14.690244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:14.702754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb7c47-f271-4dc4-b2ad-f7c299237b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:14.704025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb7c47-f271-4dc4-b2ad-f7c299237b99 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:14.705015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb7c47-f271-4dc4-b2ad-f7c299237b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:14.705661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb7c47-f271-4dc4-b2ad-f7c299237b99 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:14.709910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb7c47-f271-4dc4-b2ad-f7c299237b99 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:14.712106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb7c47-f271-4dc4-b2ad-f7c299237b99 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:14.726256 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 122/250] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:14 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:14.729057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ce2a4-566a-4cf1-8c02-801321b96999 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:14.729952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ce2a4-566a-4cf1-8c02-801321b96999 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:14.730430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ce2a4-566a-4cf1-8c02-801321b96999 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:14.734716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 123/251] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:14.737245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ce2a4-566a-4cf1-8c02-801321b96999 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:14.785774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd0b6c-f998-472a-984b-5ebf787bc9fb None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:14.829471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa74a0-144c-4f9c-b3c0-4f40cfd1e132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:14.830875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa74a0-144c-4f9c-b3c0-4f40cfd1e132 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:14.831407 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa74a0-144c-4f9c-b3c0-4f40cfd1e132 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:14.831902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa74a0-144c-4f9c-b3c0-4f40cfd1e132 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:14.838882 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa74a0-144c-4f9c-b3c0-4f40cfd1e132 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:14.840671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa74a0-144c-4f9c-b3c0-4f40cfd1e132 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:14.843946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 129/252] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:14 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:14.845443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f43b3-9a2c-4003-a41c-b7eef839f85f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:14.846175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f43b3-9a2c-4003-a41c-b7eef839f85f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:14.847258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f43b3-9a2c-4003-a41c-b7eef839f85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:14.873755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d01f43b3-9a2c-4003-a41c-b7eef839f85f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:14.874973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d01f43b3-9a2c-4003-a41c-b7eef839f85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:14.976753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 130/253] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:14.976753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f43b3-9a2c-4003-a41c-b7eef839f85f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:15.027381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ce2a4-566a-4cf1-8c02-801321b96999 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:15.035926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:15.062325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:15.109425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52844ef-8a1f-4e5b-9f06-487044dd7031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:15.110009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52844ef-8a1f-4e5b-9f06-487044dd7031 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:15.110287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52844ef-8a1f-4e5b-9f06-487044dd7031 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:15.110571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52844ef-8a1f-4e5b-9f06-487044dd7031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:15.111395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52844ef-8a1f-4e5b-9f06-487044dd7031 None admin] RBAC: Adding query filter params (role.id=45c97834bc614c5ca5122d691ce93316, scope.project.id=fea9cdff50914413b637910a718a6cd5, user.id=579c35a891a945078195650268cbf6e2) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:15.111715 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52844ef-8a1f-4e5b-9f06-487044dd7031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:15.114560 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52844ef-8a1f-4e5b-9f06-487044dd7031 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:15.122896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 124/254] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:14 2022] GET /identity/v3/role_assignments?user.id=579c35a891a945078195650268cbf6e2&scope.project.id=fea9cdff50914413b637910a718a6cd5&role.id=45c97834bc614c5ca5122d691ce93316 => generated 259 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:15.123854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52844ef-8a1f-4e5b-9f06-487044dd7031 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:15.148417 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f43b3-9a2c-4003-a41c-b7eef839f85f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:15.244951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca8710-e034-480e-93d9-9281f1f7001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:15.245942 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8710-e034-480e-93d9-9281f1f7001e None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:15.246221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8710-e034-480e-93d9-9281f1f7001e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:15.246489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8710-e034-480e-93d9-9281f1f7001e None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5/users/13fb018ff58842608881eb26b024a95a/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:15.254422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca8710-e034-480e-93d9-9281f1f7001e None admin] RBAC: Authorizing `identity:create_grant(project_id=fea9cdff50914413b637910a718a6cd5, user_id=13fb018ff58842608881eb26b024a95a, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:15.256960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca8710-e034-480e-93d9-9281f1f7001e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:15.287320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 131/255] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:03:14 2022] PUT /identity/v3/projects/fea9cdff50914413b637910a718a6cd5/users/13fb018ff58842608881eb26b024a95a/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:15.288757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40661c5-1459-4853-8726-a488c203222a None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:15.289044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40661c5-1459-4853-8726-a488c203222a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:15.289317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40661c5-1459-4853-8726-a488c203222a None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:15.298457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 132/256] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:15.324441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52844ef-8a1f-4e5b-9f06-487044dd7031 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:15.343270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b9332-a19d-40f5-af49-5087f5150d1e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:15.343595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b9332-a19d-40f5-af49-5087f5150d1e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:15.343861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b9332-a19d-40f5-af49-5087f5150d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:15.380989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1a9b9332-a19d-40f5-af49-5087f5150d1e None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:15.382234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1a9b9332-a19d-40f5-af49-5087f5150d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:15.387821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba91e1-3264-4d7d-b705-5be5c8d87064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:15.389239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba91e1-3264-4d7d-b705-5be5c8d87064 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:15.389524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba91e1-3264-4d7d-b705-5be5c8d87064 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:15.389830 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba91e1-3264-4d7d-b705-5be5c8d87064 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:15.407992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba91e1-3264-4d7d-b705-5be5c8d87064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c1ba91e1-3264-4d7d-b705-5be5c8d87064 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:15.419981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:15.420929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:15.421775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:15.421775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:15.421775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:15.421775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:15.421775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:15.421775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:15.421775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:15.421775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:15.421775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:15.421775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:15.421775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:15.421775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:15.423583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 125/257] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:03:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:15.431362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba91e1-3264-4d7d-b705-5be5c8d87064 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:15.502124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:15.572932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 133/258] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:15.613992 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b9332-a19d-40f5-af49-5087f5150d1e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:15.664110 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba91e1-3264-4d7d-b705-5be5c8d87064 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:15.731965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:15.750361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0f897-3cd4-44aa-951e-95cb31da1213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:15.751036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0f897-3cd4-44aa-951e-95cb31da1213 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:15.751312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0f897-3cd4-44aa-951e-95cb31da1213 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:15.751585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0f897-3cd4-44aa-951e-95cb31da1213 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:15.752350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0f897-3cd4-44aa-951e-95cb31da1213 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:15.752657 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0f897-3cd4-44aa-951e-95cb31da1213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:15.755179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0f897-3cd4-44aa-951e-95cb31da1213 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:15.774430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 126/259] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:15.783060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0f897-3cd4-44aa-951e-95cb31da1213 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:15.856116 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b9332-a19d-40f5-af49-5087f5150d1e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:15.939446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:15.947802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd05954-7a3c-467a-81c0-e4a497ead85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:15.949155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd05954-7a3c-467a-81c0-e4a497ead85d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:15.949846 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd05954-7a3c-467a-81c0-e4a497ead85d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:15.951127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd05954-7a3c-467a-81c0-e4a497ead85d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:15.956304 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0f897-3cd4-44aa-951e-95cb31da1213 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:15.990957 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd05954-7a3c-467a-81c0-e4a497ead85d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:15.997046 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c27f6a-c9ae-4c3f-a703-0b73092df528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:15.998258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c27f6a-c9ae-4c3f-a703-0b73092df528 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:15.998544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c27f6a-c9ae-4c3f-a703-0b73092df528 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:15.998820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c27f6a-c9ae-4c3f-a703-0b73092df528 None admin] PATH_INFO: `/v3/users/2114c1df7f3f4c7bbeaf32b42b3bd74b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:16.005763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c27f6a-c9ae-4c3f-a703-0b73092df528 None admin] RBAC: Authorizing `identity:get_user(user_id=2114c1df7f3f4c7bbeaf32b42b3bd74b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:16.005763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c27f6a-c9ae-4c3f-a703-0b73092df528 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:16.005763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 127/260] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:15 2022] GET /identity/v3/users/2114c1df7f3f4c7bbeaf32b42b3bd74b => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:16.015800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c27f6a-c9ae-4c3f-a703-0b73092df528 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-9fd05954-7a3c-467a-81c0-e4a497ead85d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:16.026951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:16.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:16.029203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:16.029203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:16.029203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:16.029203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:16.029203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:16.029203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:16.029203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:16.029203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:16.029203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:16.029203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:16.029203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:16.029203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:16.030629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 134/261] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:03:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:16.036743 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd05954-7a3c-467a-81c0-e4a497ead85d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:16.264613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd05954-7a3c-467a-81c0-e4a497ead85d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:16.281339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:16.290847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c27f6a-c9ae-4c3f-a703-0b73092df528 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:16.307732 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d7362-57d5-4abe-a8b8-69ced0da73aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:16.309055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d7362-57d5-4abe-a8b8-69ced0da73aa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:16.309338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d7362-57d5-4abe-a8b8-69ced0da73aa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:16.309615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d7362-57d5-4abe-a8b8-69ced0da73aa None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:16.316874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d7362-57d5-4abe-a8b8-69ced0da73aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:16.316874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d7362-57d5-4abe-a8b8-69ced0da73aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:16.317305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d7362-57d5-4abe-a8b8-69ced0da73aa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:16.330535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 135/262] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:16.335707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d7362-57d5-4abe-a8b8-69ced0da73aa None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:16.395190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3fb6e-3a64-4be4-a473-97bddf565852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:16.395858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3fb6e-3a64-4be4-a473-97bddf565852 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:16.396135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3fb6e-3a64-4be4-a473-97bddf565852 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:16.396401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3fb6e-3a64-4be4-a473-97bddf565852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:16.476678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:16.504715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d7362-57d5-4abe-a8b8-69ced0da73aa None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:16.553869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fe40d-a50d-4446-87d4-eac192f7a4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:16.554674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fe40d-a50d-4446-87d4-eac192f7a4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:16.554953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fe40d-a50d-4446-87d4-eac192f7a4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:16.555224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fe40d-a50d-4446-87d4-eac192f7a4f2 None admin] PATH_INFO: `/v3/users/2114c1df7f3f4c7bbeaf32b42b3bd74b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:16.557481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fe40d-a50d-4446-87d4-eac192f7a4f2 None admin] RBAC: Authorizing `identity:get_user(user_id=2114c1df7f3f4c7bbeaf32b42b3bd74b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:16.558980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fe40d-a50d-4446-87d4-eac192f7a4f2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:16.561192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 136/263] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:16 2022] GET /identity/v3/users/2114c1df7f3f4c7bbeaf32b42b3bd74b => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:16.567888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fe40d-a50d-4446-87d4-eac192f7a4f2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:16.599064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3fb6e-3a64-4be4-a473-97bddf565852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:16.601344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3fb6e-3a64-4be4-a473-97bddf565852 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:16.667934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:16.730041 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fe40d-a50d-4446-87d4-eac192f7a4f2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:16.768898 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0386fae-a72b-4b19-8bc2-72e369ec795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:16.769819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0386fae-a72b-4b19-8bc2-72e369ec795b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:16.770245 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0386fae-a72b-4b19-8bc2-72e369ec795b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:16.770688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0386fae-a72b-4b19-8bc2-72e369ec795b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:16.827406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 128/264] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 811 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:16.832104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3fb6e-3a64-4be4-a473-97bddf565852 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:16.847417 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0386fae-a72b-4b19-8bc2-72e369ec795b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:16.849160 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0386fae-a72b-4b19-8bc2-72e369ec795b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:16.897201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:16.960357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 137/265] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:16.964940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:16.967002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0386fae-a72b-4b19-8bc2-72e369ec795b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:17.073503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3fb6e-3a64-4be4-a473-97bddf565852 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:17.090263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0386fae-a72b-4b19-8bc2-72e369ec795b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:17.128106 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23634e06-289b-49bc-a1f8-d27900aa8398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:17.129246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23634e06-289b-49bc-a1f8-d27900aa8398 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:17.129696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23634e06-289b-49bc-a1f8-d27900aa8398 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:17.130163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23634e06-289b-49bc-a1f8-d27900aa8398 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:17.131503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23634e06-289b-49bc-a1f8-d27900aa8398 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:17.133141 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23634e06-289b-49bc-a1f8-d27900aa8398 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:17.135033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 138/266] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:16 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:17.141609 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23634e06-289b-49bc-a1f8-d27900aa8398 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:17.154997 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac669d6c-56d6-47b4-905d-0a191b7d5a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:17.155994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac669d6c-56d6-47b4-905d-0a191b7d5a50 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:17.156404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac669d6c-56d6-47b4-905d-0a191b7d5a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:17.156749 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac669d6c-56d6-47b4-905d-0a191b7d5a50 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:17.157075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:17.158468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac669d6c-56d6-47b4-905d-0a191b7d5a50 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:17.160253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac669d6c-56d6-47b4-905d-0a191b7d5a50 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:17.162333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 129/267] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:16 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:17.170792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac669d6c-56d6-47b4-905d-0a191b7d5a50 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:17.372854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23634e06-289b-49bc-a1f8-d27900aa8398 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:17.421969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac669d6c-56d6-47b4-905d-0a191b7d5a50 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:17.451644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55047a-c5a8-40b3-8751-41609445039c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:17.452424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55047a-c5a8-40b3-8751-41609445039c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:17.452703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55047a-c5a8-40b3-8751-41609445039c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:17.452974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55047a-c5a8-40b3-8751-41609445039c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:17.453822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55047a-c5a8-40b3-8751-41609445039c None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=fea9cdff50914413b637910a718a6cd5, user.id=2114c1df7f3f4c7bbeaf32b42b3bd74b) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:17.454258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55047a-c5a8-40b3-8751-41609445039c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:17.454584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:17.461980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55047a-c5a8-40b3-8751-41609445039c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:17.463777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410a178-532c-404b-8ec3-553316310e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:17.464435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410a178-532c-404b-8ec3-553316310e8d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:17.464714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410a178-532c-404b-8ec3-553316310e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:17.464985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410a178-532c-404b-8ec3-553316310e8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:17.465842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410a178-532c-404b-8ec3-553316310e8d None admin] RBAC: Adding query filter params (role.id=506b3cf33324478abe0f72a9d408aa54, scope.project.id=fea9cdff50914413b637910a718a6cd5, user.id=2114c1df7f3f4c7bbeaf32b42b3bd74b) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:17.466692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410a178-532c-404b-8ec3-553316310e8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:17.468375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410a178-532c-404b-8ec3-553316310e8d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:17.475046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 139/269] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:17 2022] GET /identity/v3/role_assignments?user.id=2114c1df7f3f4c7bbeaf32b42b3bd74b&scope.project.id=fea9cdff50914413b637910a718a6cd5&role.id=584beecb022b422f820e77f7623ada5e => generated 608 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:17.482464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 130/269] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:17 2022] GET /identity/v3/role_assignments?user.id=2114c1df7f3f4c7bbeaf32b42b3bd74b&scope.project.id=fea9cdff50914413b637910a718a6cd5&role.id=506b3cf33324478abe0f72a9d408aa54 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:17.489692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef9e6c-c812-4eb5-b33c-e4a3bb0a440e None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:17.490014 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef9e6c-c812-4eb5-b33c-e4a3bb0a440e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:17.490293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef9e6c-c812-4eb5-b33c-e4a3bb0a440e None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:17.492066 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 140/270] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:17.498819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764a8f3-2c41-4447-a514-29d514ec5cb7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:17.499340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764a8f3-2c41-4447-a514-29d514ec5cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:17.499752 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764a8f3-2c41-4447-a514-29d514ec5cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:17.527010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d764a8f3-2c41-4447-a514-29d514ec5cb7 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:17.528488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d764a8f3-2c41-4447-a514-29d514ec5cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:17.639000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 131/271] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:17.676899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef9e6c-c812-4eb5-b33c-e4a3bb0a440e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:17.817489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059eb36-fcaa-48e1-b53f-1da93cc38474 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:17.817971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059eb36-fcaa-48e1-b53f-1da93cc38474 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:17.818351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059eb36-fcaa-48e1-b53f-1da93cc38474 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:17.819252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:17.820085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 132/272] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:03:17.825610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9fb72-d867-4483-9151-f88d691a3405 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:17.825917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9fb72-d867-4483-9151-f88d691a3405 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:17.826210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9fb72-d867-4483-9151-f88d691a3405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:17.864557 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-84d9fb72-d867-4483-9151-f88d691a3405 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:17.864557 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-84d9fb72-d867-4483-9151-f88d691a3405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:17.891852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef9e6c-c812-4eb5-b33c-e4a3bb0a440e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:17.968690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8b9e4-f50f-47da-9cf6-dbdf6be60865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:17.970223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8b9e4-f50f-47da-9cf6-dbdf6be60865 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:17.970722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8b9e4-f50f-47da-9cf6-dbdf6be60865 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:17.971204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8b9e4-f50f-47da-9cf6-dbdf6be60865 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:17.991572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8b9e4-f50f-47da-9cf6-dbdf6be60865 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:17.996904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 133/273] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-c7e8b9e4-f50f-47da-9cf6-dbdf6be60865 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:17.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:17.999990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:18.001188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 141/274] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:18.006804 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9fb72-d867-4483-9151-f88d691a3405 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:18.030622 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8b9e4-f50f-47da-9cf6-dbdf6be60865 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:18.101492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:18.170875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9fb72-d867-4483-9151-f88d691a3405 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:18.208701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8b9e4-f50f-47da-9cf6-dbdf6be60865 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:18.213435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf321b-d5da-4a97-ad10-f2d777676ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:18.214267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf321b-d5da-4a97-ad10-f2d777676ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:18.214648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf321b-d5da-4a97-ad10-f2d777676ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:18.215043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf321b-d5da-4a97-ad10-f2d777676ef5 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:18.215904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf321b-d5da-4a97-ad10-f2d777676ef5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:18.216353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf321b-d5da-4a97-ad10-f2d777676ef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:18.217900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf321b-d5da-4a97-ad10-f2d777676ef5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:18.227131 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 134/275] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:18.233763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf321b-d5da-4a97-ad10-f2d777676ef5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:18.257439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e96c78-dc03-4e11-9bd2-c9a68aac7722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:18.258271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e96c78-dc03-4e11-9bd2-c9a68aac7722 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:18.258551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e96c78-dc03-4e11-9bd2-c9a68aac7722 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:18.258822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e96c78-dc03-4e11-9bd2-c9a68aac7722 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:18.260010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e96c78-dc03-4e11-9bd2-c9a68aac7722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:18.261470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e96c78-dc03-4e11-9bd2-c9a68aac7722 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:18.268675 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 142/276] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:18.275111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e96c78-dc03-4e11-9bd2-c9a68aac7722 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:18.277843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:18.350133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf321b-d5da-4a97-ad10-f2d777676ef5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:18.365098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:18.391715 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75bb5b-e4bd-47b8-8429-d482a5c606a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:18.392732 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75bb5b-e4bd-47b8-8429-d482a5c606a7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:18.393014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75bb5b-e4bd-47b8-8429-d482a5c606a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:18.393286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75bb5b-e4bd-47b8-8429-d482a5c606a7 None admin] PATH_INFO: `/v3/users/7dd580e32f474e118d540b28e284682f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:18.404385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e96c78-dc03-4e11-9bd2-c9a68aac7722 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:18.409388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75bb5b-e4bd-47b8-8429-d482a5c606a7 None admin] RBAC: Authorizing `identity:get_user(user_id=7dd580e32f474e118d540b28e284682f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:18.411312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75bb5b-e4bd-47b8-8429-d482a5c606a7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:18.413404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 135/277] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:18 2022] GET /identity/v3/users/7dd580e32f474e118d540b28e284682f => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:18.419917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75bb5b-e4bd-47b8-8429-d482a5c606a7 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:18.453033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee68aeb-ed94-4af2-811b-08d53a88eed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:18.453925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee68aeb-ed94-4af2-811b-08d53a88eed1 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:18.454255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee68aeb-ed94-4af2-811b-08d53a88eed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:18.454531 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee68aeb-ed94-4af2-811b-08d53a88eed1 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:18.455292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee68aeb-ed94-4af2-811b-08d53a88eed1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:18.457855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee68aeb-ed94-4af2-811b-08d53a88eed1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:18.488863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 143/278] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:03:18 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:18.561377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75bb5b-e4bd-47b8-8429-d482a5c606a7 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:18.611043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad708c-f913-4c87-9304-03f49b6191f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:18.612153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad708c-f913-4c87-9304-03f49b6191f8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:18.612635 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad708c-f913-4c87-9304-03f49b6191f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:18.613104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad708c-f913-4c87-9304-03f49b6191f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:18.614174 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad708c-f913-4c87-9304-03f49b6191f8 None admin] RBAC: Adding query filter params (role.id=e1a5192e7abc4046bc2c0673176ea1b6, user.id=7dd580e32f474e118d540b28e284682f, scope.system=all) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:18.614690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad708c-f913-4c87-9304-03f49b6191f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:18.616450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad708c-f913-4c87-9304-03f49b6191f8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:18.623996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 136/279] 10.222.0.46 () {56 vars in 1308 bytes} [Thu May 26 18:03:18 2022] GET /identity/v3/role_assignments?user.id=7dd580e32f474e118d540b28e284682f&scope.system=all&role.id=e1a5192e7abc4046bc2c0673176ea1b6 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:18.719375 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff290e8f-b00a-4753-9691-9945237a2b4c None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:18.719991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff290e8f-b00a-4753-9691-9945237a2b4c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:18.720483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff290e8f-b00a-4753-9691-9945237a2b4c None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:18.722148 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 144/280] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:18.728211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214566ac-2afe-481c-b51c-31d8b6e76c70 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:18.728812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214566ac-2afe-481c-b51c-31d8b6e76c70 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:18.729287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214566ac-2afe-481c-b51c-31d8b6e76c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:18.729389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d1392-44f9-46de-9486-7a5fcf7f67c1 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:18.729672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d1392-44f9-46de-9486-7a5fcf7f67c1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:18.729981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d1392-44f9-46de-9486-7a5fcf7f67c1 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:18.732497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 145/281] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:03:18.738811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff8fbd-498f-4ea7-b44b-a70ebb3eb38f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:18.739424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff8fbd-498f-4ea7-b44b-a70ebb3eb38f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:18.739914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff8fbd-498f-4ea7-b44b-a70ebb3eb38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:18.748969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-214566ac-2afe-481c-b51c-31d8b6e76c70 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:18.750301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-214566ac-2afe-481c-b51c-31d8b6e76c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:18.760022 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a8ff8fbd-498f-4ea7-b44b-a70ebb3eb38f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:18.761440 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a8ff8fbd-498f-4ea7-b44b-a70ebb3eb38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:18.790790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:18.831382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:18.848536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 137/282] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:18.856235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 146/283] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:18.872690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214566ac-2afe-481c-b51c-31d8b6e76c70 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:18.883318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff8fbd-498f-4ea7-b44b-a70ebb3eb38f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:18.988758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214566ac-2afe-481c-b51c-31d8b6e76c70 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:18.994188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff8fbd-498f-4ea7-b44b-a70ebb3eb38f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:19.026829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719714b6-f319-4526-bca6-fcef90a9f940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:19.027965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719714b6-f319-4526-bca6-fcef90a9f940 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:19.029000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719714b6-f319-4526-bca6-fcef90a9f940 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:19.029465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719714b6-f319-4526-bca6-fcef90a9f940 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:19.037632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ac36c-e2af-45bd-ba3a-fd379f8b20d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:19.038772 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ac36c-e2af-45bd-ba3a-fd379f8b20d6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:19.039190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ac36c-e2af-45bd-ba3a-fd379f8b20d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:19.039598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ac36c-e2af-45bd-ba3a-fd379f8b20d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:19.041435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719714b6-f319-4526-bca6-fcef90a9f940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-719714b6-f319-4526-bca6-fcef90a9f940 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:19.050041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:19.050998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:19.051778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:19.051778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:19.051778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:19.051778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:19.051778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:19.051778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:19.051778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:19.051778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:19.051778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:19.051778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:19.051778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:19.051778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:19.052362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 138/284] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:19.055631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ac36c-e2af-45bd-ba3a-fd379f8b20d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:19.057751 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719714b6-f319-4526-bca6-fcef90a9f940 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-175ac36c-e2af-45bd-ba3a-fd379f8b20d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:19.065406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:19.066326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:19.067092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:19.067092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:19.067092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:19.067092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:19.067092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:19.067092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:19.067092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:19.067092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:19.067092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:19.067092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:19.067092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:19.067092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:19.069418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 147/285] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:19.075625 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ac36c-e2af-45bd-ba3a-fd379f8b20d6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:19.087504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:19.133424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:19.174947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719714b6-f319-4526-bca6-fcef90a9f940 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:19.192154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ac36c-e2af-45bd-ba3a-fd379f8b20d6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:19.212060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e2e70-4bd8-4e43-9b01-08d8aa1e8058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:19.212824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e2e70-4bd8-4e43-9b01-08d8aa1e8058 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:19.213178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e2e70-4bd8-4e43-9b01-08d8aa1e8058 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:19.213513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e2e70-4bd8-4e43-9b01-08d8aa1e8058 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:19.214372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e2e70-4bd8-4e43-9b01-08d8aa1e8058 None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:19.214740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e2e70-4bd8-4e43-9b01-08d8aa1e8058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:19.216373 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e2e70-4bd8-4e43-9b01-08d8aa1e8058 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:19.225930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 139/286] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:19.230778 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f719a54-4d6f-44ac-99b8-a947835d4e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:19.231491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e2e70-4bd8-4e43-9b01-08d8aa1e8058 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:19.233184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f719a54-4d6f-44ac-99b8-a947835d4e12 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:19.233718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f719a54-4d6f-44ac-99b8-a947835d4e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:19.234187 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f719a54-4d6f-44ac-99b8-a947835d4e12 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:19.235239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f719a54-4d6f-44ac-99b8-a947835d4e12 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:19.235725 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f719a54-4d6f-44ac-99b8-a947835d4e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:19.237439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f719a54-4d6f-44ac-99b8-a947835d4e12 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:19.246806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 148/287] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:19.254334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f719a54-4d6f-44ac-99b8-a947835d4e12 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:19.365328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f719a54-4d6f-44ac-99b8-a947835d4e12 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:19.386988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:19.393348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:19.404908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca5cfb-00cf-497a-930f-0360e0b7f63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:19.406148 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca5cfb-00cf-497a-930f-0360e0b7f63c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:19.406633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca5cfb-00cf-497a-930f-0360e0b7f63c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:19.407075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca5cfb-00cf-497a-930f-0360e0b7f63c None admin] PATH_INFO: `/v3/users/579c35a891a945078195650268cbf6e2` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:19.408665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca5cfb-00cf-497a-930f-0360e0b7f63c None admin] RBAC: Authorizing `identity:get_user(user_id=579c35a891a945078195650268cbf6e2)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:19.410642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca5cfb-00cf-497a-930f-0360e0b7f63c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:19.413024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 149/288] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:19 2022] GET /identity/v3/users/579c35a891a945078195650268cbf6e2 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:19.418428 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca5cfb-00cf-497a-930f-0360e0b7f63c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:19.424587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e2e70-4bd8-4e43-9b01-08d8aa1e8058 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:19.485364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42395ca4-747d-43d6-8aa3-d25aa705a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:19.486569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42395ca4-747d-43d6-8aa3-d25aa705a8ca None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:19.487738 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42395ca4-747d-43d6-8aa3-d25aa705a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:19.489542 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42395ca4-747d-43d6-8aa3-d25aa705a8ca None admin] PATH_INFO: `/v3/users/13fb018ff58842608881eb26b024a95a` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:19.491851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42395ca4-747d-43d6-8aa3-d25aa705a8ca None admin] RBAC: Authorizing `identity:get_user(user_id=13fb018ff58842608881eb26b024a95a)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:19.493490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42395ca4-747d-43d6-8aa3-d25aa705a8ca None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:19.501299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 140/289] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:19 2022] GET /identity/v3/users/13fb018ff58842608881eb26b024a95a => generated 297 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:19.503531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42395ca4-747d-43d6-8aa3-d25aa705a8ca None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:19.540689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca5cfb-00cf-497a-930f-0360e0b7f63c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:19.576809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c1862-9961-4206-bd85-79f4379483eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:19.578073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c1862-9961-4206-bd85-79f4379483eb None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:19.578550 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c1862-9961-4206-bd85-79f4379483eb None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:19.579024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c1862-9961-4206-bd85-79f4379483eb None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:19.580439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c1862-9961-4206-bd85-79f4379483eb None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:19.582093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c1862-9961-4206-bd85-79f4379483eb None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:19.584274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 150/290] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:19 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:19.589868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c1862-9961-4206-bd85-79f4379483eb None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:19.654116 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:19.711718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42395ca4-747d-43d6-8aa3-d25aa705a8ca None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:19.716401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c1862-9961-4206-bd85-79f4379483eb None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:19.783207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1df830-383e-45ed-84d8-5754367eef0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:19.785377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1df830-383e-45ed-84d8-5754367eef0e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:19.786286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1df830-383e-45ed-84d8-5754367eef0e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:19.787080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1df830-383e-45ed-84d8-5754367eef0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:19.788960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77450602-5cb1-43fb-bb05-20186744731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:19.792117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77450602-5cb1-43fb-bb05-20186744731c None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:19.792616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77450602-5cb1-43fb-bb05-20186744731c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:19.793079 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77450602-5cb1-43fb-bb05-20186744731c None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5/users/579c35a891a945078195650268cbf6e2/roles/45c97834bc614c5ca5122d691ce93316` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:19.796597 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77450602-5cb1-43fb-bb05-20186744731c None admin] RBAC: Authorizing `identity:create_grant(project_id=fea9cdff50914413b637910a718a6cd5, user_id=579c35a891a945078195650268cbf6e2, role_id=45c97834bc614c5ca5122d691ce93316)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:19.801945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77450602-5cb1-43fb-bb05-20186744731c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:19.821765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 151/291] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:03:19 2022] PUT /identity/v3/projects/fea9cdff50914413b637910a718a6cd5/users/579c35a891a945078195650268cbf6e2/roles/45c97834bc614c5ca5122d691ce93316 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:03:19.838211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:19.899179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1df830-383e-45ed-84d8-5754367eef0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:19.900757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1df830-383e-45ed-84d8-5754367eef0e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:20.015130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 141/292] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 18:03:20.020473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77450602-5cb1-43fb-bb05-20186744731c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:20.172843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77450602-5cb1-43fb-bb05-20186744731c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:20.182993 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:20.213490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da821906-2fcc-4fcd-b898-828057ffceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:20.214740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da821906-2fcc-4fcd-b898-828057ffceb4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:20.215224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da821906-2fcc-4fcd-b898-828057ffceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:20.215697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da821906-2fcc-4fcd-b898-828057ffceb4 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:20.217107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da821906-2fcc-4fcd-b898-828057ffceb4 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:20.218812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da821906-2fcc-4fcd-b898-828057ffceb4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:20.222532 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 152/293] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:20 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:20.226582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1df830-383e-45ed-84d8-5754367eef0e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:20.302426 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:20.355771 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1df830-383e-45ed-84d8-5754367eef0e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:20.390686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9d44c-fbd8-459a-9773-08aa27e5ea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:20.391332 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9d44c-fbd8-459a-9773-08aa27e5ea4e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:20.391607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9d44c-fbd8-459a-9773-08aa27e5ea4e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:20.391876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9d44c-fbd8-459a-9773-08aa27e5ea4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:20.392699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9d44c-fbd8-459a-9773-08aa27e5ea4e None admin] RBAC: Adding query filter params (role.id=e1a5192e7abc4046bc2c0673176ea1b6, scope.project.id=fea9cdff50914413b637910a718a6cd5, user.id=13fb018ff58842608881eb26b024a95a) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:20.393004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9d44c-fbd8-459a-9773-08aa27e5ea4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:20.394495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9d44c-fbd8-459a-9773-08aa27e5ea4e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:20.400941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 142/294] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:20 2022] GET /identity/v3/role_assignments?user.id=13fb018ff58842608881eb26b024a95a&scope.project.id=fea9cdff50914413b637910a718a6cd5&role.id=e1a5192e7abc4046bc2c0673176ea1b6 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:21.303927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6817b6b-de7f-4b06-84dc-bf61662a8661 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:21.304450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6817b6b-de7f-4b06-84dc-bf61662a8661 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:21.304450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6817b6b-de7f-4b06-84dc-bf61662a8661 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:21.306280 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 153/295] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:21.312842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6637288-f937-4024-b55e-476169b85160 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:21.313831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6637288-f937-4024-b55e-476169b85160 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:21.314337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6637288-f937-4024-b55e-476169b85160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:21.333060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d6637288-f937-4024-b55e-476169b85160 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:21.334269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d6637288-f937-4024-b55e-476169b85160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:21.343670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c593825-7b02-4c34-8435-39bdf44dd0ba None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:21.344256 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c593825-7b02-4c34-8435-39bdf44dd0ba None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:21.344753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c593825-7b02-4c34-8435-39bdf44dd0ba None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:21.346741 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 154/296] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:03:21.353359 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925e312-5bc8-4579-a53a-348de95aceb2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:21.353649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925e312-5bc8-4579-a53a-348de95aceb2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:21.353944 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925e312-5bc8-4579-a53a-348de95aceb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:21.375651 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c925e312-5bc8-4579-a53a-348de95aceb2 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:21.377211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c925e312-5bc8-4579-a53a-348de95aceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:21.422494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 143/297] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:21.445306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6637288-f937-4024-b55e-476169b85160 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:21.453388 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 155/298] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:21.475396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925e312-5bc8-4579-a53a-348de95aceb2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:21.502935 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:21.563239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6637288-f937-4024-b55e-476169b85160 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:21.581940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925e312-5bc8-4579-a53a-348de95aceb2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:21.585301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:21.600038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17551c37-3c86-4380-b9b0-74d3fbf9639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:21.600829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17551c37-3c86-4380-b9b0-74d3fbf9639d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:21.601117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17551c37-3c86-4380-b9b0-74d3fbf9639d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:21.601390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17551c37-3c86-4380-b9b0-74d3fbf9639d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:21.612530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17551c37-3c86-4380-b9b0-74d3fbf9639d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:21.618806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506dc823-c0b9-4c57-993d-2a8adb3e16fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:21.619589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506dc823-c0b9-4c57-993d-2a8adb3e16fd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:21.619869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506dc823-c0b9-4c57-993d-2a8adb3e16fd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:21.620136 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506dc823-c0b9-4c57-993d-2a8adb3e16fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-17551c37-3c86-4380-b9b0-74d3fbf9639d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:21.620289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:21.621115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:21.621949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 144/299] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:21.623872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6db2f-c15b-4623-8500-b5c01a28c4f7 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:21.623872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6db2f-c15b-4623-8500-b5c01a28c4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:21.623872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6db2f-c15b-4623-8500-b5c01a28c4f7 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:21.625179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 145/300] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:21.630053 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6db2f-c15b-4623-8500-b5c01a28c4f7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:21.637899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506dc823-c0b9-4c57-993d-2a8adb3e16fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-506dc823-c0b9-4c57-993d-2a8adb3e16fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:21.648363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:21.649212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:21.650100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:21.650100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:21.650100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:21.650100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:21.650100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:21.650100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:21.650100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:21.650100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:21.650100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:21.650100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:21.650100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:21.650100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:21.650688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 156/301] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:21.652425 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef25ab0-1548-49d5-ad78-4a66a5bba161 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:21.652913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef25ab0-1548-49d5-ad78-4a66a5bba161 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:21.653194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef25ab0-1548-49d5-ad78-4a66a5bba161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:21.671067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-aef25ab0-1548-49d5-ad78-4a66a5bba161 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:21.672250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-aef25ab0-1548-49d5-ad78-4a66a5bba161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:21.749757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 157/302] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:21.749757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef25ab0-1548-49d5-ad78-4a66a5bba161 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:21.750868 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6db2f-c15b-4623-8500-b5c01a28c4f7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:21.761117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:21.789749 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a21a4-6cc5-412e-b95a-18bc334cd5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:21.790648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a21a4-6cc5-412e-b95a-18bc334cd5cd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:21.790934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a21a4-6cc5-412e-b95a-18bc334cd5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:21.791206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a21a4-6cc5-412e-b95a-18bc334cd5cd None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:21.792106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a21a4-6cc5-412e-b95a-18bc334cd5cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:21.792422 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a21a4-6cc5-412e-b95a-18bc334cd5cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:21.794174 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a21a4-6cc5-412e-b95a-18bc334cd5cd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:21.808226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 146/303] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:21.808226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a21a4-6cc5-412e-b95a-18bc334cd5cd None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:21.851904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:21.860762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef25ab0-1548-49d5-ad78-4a66a5bba161 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:21.897956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697819c1-4a76-401b-b672-c54d1aef60e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:21.898653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697819c1-4a76-401b-b672-c54d1aef60e7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:21.898931 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697819c1-4a76-401b-b672-c54d1aef60e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:21.899205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697819c1-4a76-401b-b672-c54d1aef60e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:21.899956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697819c1-4a76-401b-b672-c54d1aef60e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:21.900263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697819c1-4a76-401b-b672-c54d1aef60e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:21.901634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697819c1-4a76-401b-b672-c54d1aef60e7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:21.914767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 158/304] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:21.914767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697819c1-4a76-401b-b672-c54d1aef60e7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:21.922878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a21a4-6cc5-412e-b95a-18bc334cd5cd None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:21.963275 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4177b6a-e8a4-4ba0-b624-cfc70232a897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:21.964070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4177b6a-e8a4-4ba0-b624-cfc70232a897 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:21.964350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4177b6a-e8a4-4ba0-b624-cfc70232a897 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:21.964620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4177b6a-e8a4-4ba0-b624-cfc70232a897 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:21.965808 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4177b6a-e8a4-4ba0-b624-cfc70232a897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:21.967800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4177b6a-e8a4-4ba0-b624-cfc70232a897 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:21.969919 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 147/305] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:21.975011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4177b6a-e8a4-4ba0-b624-cfc70232a897 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:22.020124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:22.020563 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697819c1-4a76-401b-b672-c54d1aef60e7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:22.055883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422ffa6-2388-4bc5-895d-be50d8ae3fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:22.056753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422ffa6-2388-4bc5-895d-be50d8ae3fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:22.057032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422ffa6-2388-4bc5-895d-be50d8ae3fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:22.057301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422ffa6-2388-4bc5-895d-be50d8ae3fb5 None admin] PATH_INFO: `/v3/users/d5e38a5c2cf44769ab311bdc6af97c92` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:22.072687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422ffa6-2388-4bc5-895d-be50d8ae3fb5 None admin] RBAC: Authorizing `identity:get_user(user_id=d5e38a5c2cf44769ab311bdc6af97c92)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:22.074176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422ffa6-2388-4bc5-895d-be50d8ae3fb5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:22.076904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 159/306] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:21 2022] GET /identity/v3/users/d5e38a5c2cf44769ab311bdc6af97c92 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:22.082337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422ffa6-2388-4bc5-895d-be50d8ae3fb5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:22.085413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4177b6a-e8a4-4ba0-b624-cfc70232a897 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:22.098818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:22.121708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e713a-50bb-4b56-8f3a-6eafef0b0ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:22.122561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e713a-50bb-4b56-8f3a-6eafef0b0ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:22.122833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e713a-50bb-4b56-8f3a-6eafef0b0ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:22.123163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e713a-50bb-4b56-8f3a-6eafef0b0ce3 None admin] PATH_INFO: `/v3/users/7dd580e32f474e118d540b28e284682f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:22.124370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e713a-50bb-4b56-8f3a-6eafef0b0ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=7dd580e32f474e118d540b28e284682f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:22.125863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e713a-50bb-4b56-8f3a-6eafef0b0ce3 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:22.127485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 148/307] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:21 2022] GET /identity/v3/users/7dd580e32f474e118d540b28e284682f => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:22.128242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e713a-50bb-4b56-8f3a-6eafef0b0ce3 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:22.194999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422ffa6-2388-4bc5-895d-be50d8ae3fb5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:22.233002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6d768-9270-4de4-9499-fcc4b02c8278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:22.233585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6d768-9270-4de4-9499-fcc4b02c8278 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:22.233893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6d768-9270-4de4-9499-fcc4b02c8278 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:22.234201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6d768-9270-4de4-9499-fcc4b02c8278 None admin] PATH_INFO: `/v3/groups` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:22.234965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6d768-9270-4de4-9499-fcc4b02c8278 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:22.236440 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6d768-9270-4de4-9499-fcc4b02c8278 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:22.237273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e713a-50bb-4b56-8f3a-6eafef0b0ce3 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:22.247012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:22.254011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 160/308] 10.222.0.46 () {60 vars in 1134 bytes} [Thu May 26 18:03:22 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:22.264360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6d768-9270-4de4-9499-fcc4b02c8278 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:22.289700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b256783-366c-45fd-8095-c6e627472d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:22.291874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b256783-366c-45fd-8095-c6e627472d5c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:22.291874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b256783-366c-45fd-8095-c6e627472d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:22.291874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b256783-366c-45fd-8095-c6e627472d5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:22.292885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b256783-366c-45fd-8095-c6e627472d5c None admin] RBAC: Adding query filter params (role.id=45c97834bc614c5ca5122d691ce93316, user.id=d5e38a5c2cf44769ab311bdc6af97c92, scope.system=all) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:22.293258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b256783-366c-45fd-8095-c6e627472d5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:22.295604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b256783-366c-45fd-8095-c6e627472d5c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:22.301998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 149/309] 10.222.0.46 () {56 vars in 1308 bytes} [Thu May 26 18:03:22 2022] GET /identity/v3/role_assignments?user.id=d5e38a5c2cf44769ab311bdc6af97c92&scope.system=all&role.id=45c97834bc614c5ca5122d691ce93316 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:22.380706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:22.419171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6d768-9270-4de4-9499-fcc4b02c8278 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:22.456435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cb70c-07a5-423f-b718-4c6ca70f2eee None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:22.456968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cb70c-07a5-423f-b718-4c6ca70f2eee None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:22.456968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cb70c-07a5-423f-b718-4c6ca70f2eee None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:22.458456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 150/310] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:22.464573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13befd-6320-4ccc-ab13-4c34cabd392e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:22.464913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13befd-6320-4ccc-ab13-4c34cabd392e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:22.465115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13befd-6320-4ccc-ab13-4c34cabd392e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:22.468835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac473a-5ea2-444c-844c-63c7c7ffaf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:22.470210 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac473a-5ea2-444c-844c-63c7c7ffaf6a None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:22.470706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac473a-5ea2-444c-844c-63c7c7ffaf6a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:22.471564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac473a-5ea2-444c-844c-63c7c7ffaf6a None admin] PATH_INFO: `/v3/system/users/7dd580e32f474e118d540b28e284682f/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:22.474662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac473a-5ea2-444c-844c-63c7c7ffaf6a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7dd580e32f474e118d540b28e284682f, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:22.476448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac473a-5ea2-444c-844c-63c7c7ffaf6a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:22.485796 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8d13befd-6320-4ccc-ab13-4c34cabd392e None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:22.486728 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8d13befd-6320-4ccc-ab13-4c34cabd392e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:22.489226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 161/311] 10.222.0.46 () {58 vars in 1334 bytes} [Thu May 26 18:03:22 2022] PUT /identity/v3/system/users/7dd580e32f474e118d540b28e284682f/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:22.578709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 151/312] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:22.602114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac473a-5ea2-444c-844c-63c7c7ffaf6a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:22.724147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac473a-5ea2-444c-844c-63c7c7ffaf6a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:22.760062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714642a-2a5c-4095-bf59-a3ff1a05804f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:22.760854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714642a-2a5c-4095-bf59-a3ff1a05804f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:22.761133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714642a-2a5c-4095-bf59-a3ff1a05804f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:22.761403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714642a-2a5c-4095-bf59-a3ff1a05804f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:22.769712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:22.774460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714642a-2a5c-4095-bf59-a3ff1a05804f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f714642a-2a5c-4095-bf59-a3ff1a05804f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:22.782984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:22.784584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:22.786363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:22.786363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:22.786363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:22.786363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:22.786363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:22.786363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:22.786363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:22.786363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:22.786363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:22.786363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:22.786363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:22.786363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:22.789482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 162/313] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:22.795668 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13befd-6320-4ccc-ab13-4c34cabd392e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:22.905200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:22.916914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13befd-6320-4ccc-ab13-4c34cabd392e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:22.954032 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45eb3d2-3ee8-4054-9338-ba3784382e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:22.954711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45eb3d2-3ee8-4054-9338-ba3784382e39 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:22.954991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45eb3d2-3ee8-4054-9338-ba3784382e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:22.955268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45eb3d2-3ee8-4054-9338-ba3784382e39 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:22.956014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45eb3d2-3ee8-4054-9338-ba3784382e39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:22.956321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45eb3d2-3ee8-4054-9338-ba3784382e39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:22.957722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45eb3d2-3ee8-4054-9338-ba3784382e39 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:22.968466 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 152/314] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:22.974119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714642a-2a5c-4095-bf59-a3ff1a05804f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:23.085676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714642a-2a5c-4095-bf59-a3ff1a05804f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:23.122023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97531b8d-a47f-4866-bfb7-c62f1bd32581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:23.123968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97531b8d-a47f-4866-bfb7-c62f1bd32581 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:23.124490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97531b8d-a47f-4866-bfb7-c62f1bd32581 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:23.124963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97531b8d-a47f-4866-bfb7-c62f1bd32581 None admin] PATH_INFO: `/v3/users/579c35a891a945078195650268cbf6e2` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:23.126506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97531b8d-a47f-4866-bfb7-c62f1bd32581 None admin] RBAC: Authorizing `identity:get_user(user_id=579c35a891a945078195650268cbf6e2)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:23.128175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97531b8d-a47f-4866-bfb7-c62f1bd32581 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:23.130055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 163/315] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:22 2022] GET /identity/v3/users/579c35a891a945078195650268cbf6e2 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:23.138824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45eb3d2-3ee8-4054-9338-ba3784382e39 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:23.249391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45eb3d2-3ee8-4054-9338-ba3784382e39 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:23.284628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf12d04-c0b1-4fa1-bcc2-aa926c5013b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:23.285528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf12d04-c0b1-4fa1-bcc2-aa926c5013b6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:23.285835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf12d04-c0b1-4fa1-bcc2-aa926c5013b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:23.286110 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf12d04-c0b1-4fa1-bcc2-aa926c5013b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:23.317920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:23.356442 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf12d04-c0b1-4fa1-bcc2-aa926c5013b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:23.358554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf12d04-c0b1-4fa1-bcc2-aa926c5013b6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:23.459320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 153/316] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:23.467327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97531b8d-a47f-4866-bfb7-c62f1bd32581 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:23.534019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:23.571625 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97531b8d-a47f-4866-bfb7-c62f1bd32581 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:23.609290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f941e-4389-434e-acf2-af980b09b659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:23.610177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f941e-4389-434e-acf2-af980b09b659 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:23.610463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f941e-4389-434e-acf2-af980b09b659 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:23.610746 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f941e-4389-434e-acf2-af980b09b659 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:23.611987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f941e-4389-434e-acf2-af980b09b659 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:23.613436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f941e-4389-434e-acf2-af980b09b659 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:23.615231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 164/317] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:23 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:23.621010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf12d04-c0b1-4fa1-bcc2-aa926c5013b6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:23.683288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:23.723981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf12d04-c0b1-4fa1-bcc2-aa926c5013b6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:23.758077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258af4b5-7f0d-407e-a0d2-33f88985ef0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:23.758751 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258af4b5-7f0d-407e-a0d2-33f88985ef0a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:23.759033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258af4b5-7f0d-407e-a0d2-33f88985ef0a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:23.759306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258af4b5-7f0d-407e-a0d2-33f88985ef0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:23.760151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258af4b5-7f0d-407e-a0d2-33f88985ef0a None admin] RBAC: Adding query filter params (role.id=45c97834bc614c5ca5122d691ce93316, scope.project.id=fea9cdff50914413b637910a718a6cd5, user.id=579c35a891a945078195650268cbf6e2) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:23.760460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258af4b5-7f0d-407e-a0d2-33f88985ef0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:23.762013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258af4b5-7f0d-407e-a0d2-33f88985ef0a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:23.770083 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 154/318] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:23 2022] GET /identity/v3/role_assignments?user.id=579c35a891a945078195650268cbf6e2&scope.project.id=fea9cdff50914413b637910a718a6cd5&role.id=45c97834bc614c5ca5122d691ce93316 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:24.721521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af7c34-c2d2-4049-8cfd-390d044eaabd None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:24.722238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af7c34-c2d2-4049-8cfd-390d044eaabd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:24.722238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af7c34-c2d2-4049-8cfd-390d044eaabd None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:24.723715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 165/319] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:24.727628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43828b7b-bb48-417a-b3f8-e34aaa63e5b6 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:24.728208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43828b7b-bb48-417a-b3f8-e34aaa63e5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:24.728208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43828b7b-bb48-417a-b3f8-e34aaa63e5b6 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:24.729217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa3031-0fee-4166-96f5-da5dad32d898 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:24.729508 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa3031-0fee-4166-96f5-da5dad32d898 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:24.733765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 155/320] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:24.733970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa3031-0fee-4166-96f5-da5dad32d898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:24.735942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea525e-377b-492f-a0ce-b21f60db6eba None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:24.736494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea525e-377b-492f-a0ce-b21f60db6eba None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:24.736770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea525e-377b-492f-a0ce-b21f60db6eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:24.749470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-08aa3031-0fee-4166-96f5-da5dad32d898 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:24.750689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-08aa3031-0fee-4166-96f5-da5dad32d898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:24.756540 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-41ea525e-377b-492f-a0ce-b21f60db6eba None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:24.757717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-41ea525e-377b-492f-a0ce-b21f60db6eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:24.819461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 166/321] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:24.833757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 156/322] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:24.842737 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa3031-0fee-4166-96f5-da5dad32d898 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:24.853515 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea525e-377b-492f-a0ce-b21f60db6eba None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:24.907796 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:24.916518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:24.946092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa3031-0fee-4166-96f5-da5dad32d898 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:24.960623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea525e-377b-492f-a0ce-b21f60db6eba None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:24.980004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a547eeb-1215-4534-b105-00578c5efbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:24.980722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a547eeb-1215-4534-b105-00578c5efbfc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:24.980998 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a547eeb-1215-4534-b105-00578c5efbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:24.981266 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a547eeb-1215-4534-b105-00578c5efbfc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:24.982607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a547eeb-1215-4534-b105-00578c5efbfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:24.984041 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a547eeb-1215-4534-b105-00578c5efbfc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:24.985638 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 167/323] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:24.986824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009e9a1-a7a8-4331-9aa4-6a97c7752e47 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:24.987103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009e9a1-a7a8-4331-9aa4-6a97c7752e47 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:24.987379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009e9a1-a7a8-4331-9aa4-6a97c7752e47 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:24.988750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 168/324] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:24.991678 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009e9a1-a7a8-4331-9aa4-6a97c7752e47 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:24.996400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8ec8c-79bc-4130-b0fb-e0308a60df31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:24.997692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8ec8c-79bc-4130-b0fb-e0308a60df31 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:25.003928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8ec8c-79bc-4130-b0fb-e0308a60df31 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:25.005622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8ec8c-79bc-4130-b0fb-e0308a60df31 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:25.017691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8ec8c-79bc-4130-b0fb-e0308a60df31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-69a8ec8c-79bc-4130-b0fb-e0308a60df31 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:25.025361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:25.026324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:25.027134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 157/325] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:25.028491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4a795-e742-461b-a710-d88b73b30bd0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:25.028491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4a795-e742-461b-a710-d88b73b30bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:25.028726 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4a795-e742-461b-a710-d88b73b30bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:25.050837 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-eea4a795-e742-461b-a710-d88b73b30bd0 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:25.052017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-eea4a795-e742-461b-a710-d88b73b30bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:25.096327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009e9a1-a7a8-4331-9aa4-6a97c7752e47 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:25.117973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 158/326] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:25.118811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4a795-e742-461b-a710-d88b73b30bd0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:25.126665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:25.133216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723af835-5122-4219-8ce5-60a170880e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:25.133921 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723af835-5122-4219-8ce5-60a170880e43 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:25.134204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723af835-5122-4219-8ce5-60a170880e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:25.134472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723af835-5122-4219-8ce5-60a170880e43 None admin] PATH_INFO: `/v3/groups` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:25.135152 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723af835-5122-4219-8ce5-60a170880e43 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:25.136583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723af835-5122-4219-8ce5-60a170880e43 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:25.158909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 169/327] 10.222.0.46 () {60 vars in 1134 bytes} [Thu May 26 18:03:24 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:25.158909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723af835-5122-4219-8ce5-60a170880e43 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:25.172219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:25.237177 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4a795-e742-461b-a710-d88b73b30bd0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:25.277055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723af835-5122-4219-8ce5-60a170880e43 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:25.283905 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325ec1e-ac7c-45cc-a213-0ad2f6454908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:25.284598 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325ec1e-ac7c-45cc-a213-0ad2f6454908 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:25.284879 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325ec1e-ac7c-45cc-a213-0ad2f6454908 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:25.285410 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325ec1e-ac7c-45cc-a213-0ad2f6454908 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:25.286211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325ec1e-ac7c-45cc-a213-0ad2f6454908 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:25.286522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325ec1e-ac7c-45cc-a213-0ad2f6454908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:25.287962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325ec1e-ac7c-45cc-a213-0ad2f6454908 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:25.297815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 159/328] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:25.303532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325ec1e-ac7c-45cc-a213-0ad2f6454908 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:25.322426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ca994-e9a1-40e0-a4f3-04c2907ffb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:25.323250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ca994-e9a1-40e0-a4f3-04c2907ffb24 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:25.323533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ca994-e9a1-40e0-a4f3-04c2907ffb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:25.323916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ca994-e9a1-40e0-a4f3-04c2907ffb24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:25.335894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ca994-e9a1-40e0-a4f3-04c2907ffb24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-158ca994-e9a1-40e0-a4f3-04c2907ffb24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:25.344395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:25.345271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:25.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 170/329] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:25.351214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ca994-e9a1-40e0-a4f3-04c2907ffb24 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:25.420766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:25.438692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325ec1e-ac7c-45cc-a213-0ad2f6454908 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:25.449012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:25.484472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ca994-e9a1-40e0-a4f3-04c2907ffb24 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:25.484665 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224afa85-9b19-43f7-ad71-0b3b0004767d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:25.485385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224afa85-9b19-43f7-ad71-0b3b0004767d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:25.485669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224afa85-9b19-43f7-ad71-0b3b0004767d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:25.486018 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224afa85-9b19-43f7-ad71-0b3b0004767d None admin] PATH_INFO: `/v3/users/d5e38a5c2cf44769ab311bdc6af97c92` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:25.487477 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224afa85-9b19-43f7-ad71-0b3b0004767d None admin] RBAC: Authorizing `identity:get_user(user_id=d5e38a5c2cf44769ab311bdc6af97c92)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:25.489163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224afa85-9b19-43f7-ad71-0b3b0004767d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:25.491499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 160/330] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:25 2022] GET /identity/v3/users/d5e38a5c2cf44769ab311bdc6af97c92 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:25.496590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224afa85-9b19-43f7-ad71-0b3b0004767d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:25.523439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b419dc6-e90f-40da-8402-51a60fe3e5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:25.525100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b419dc6-e90f-40da-8402-51a60fe3e5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:25.525385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b419dc6-e90f-40da-8402-51a60fe3e5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:25.525660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b419dc6-e90f-40da-8402-51a60fe3e5c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:25.526478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b419dc6-e90f-40da-8402-51a60fe3e5c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:25.526800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b419dc6-e90f-40da-8402-51a60fe3e5c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:25.528373 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b419dc6-e90f-40da-8402-51a60fe3e5c1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:25.539713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 171/331] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:25.551442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b419dc6-e90f-40da-8402-51a60fe3e5c1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:25.637150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224afa85-9b19-43f7-ad71-0b3b0004767d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:25.688558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19795e1-d3b5-4d1b-bb67-4200f81f0223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:25.689810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19795e1-d3b5-4d1b-bb67-4200f81f0223 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:25.690284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19795e1-d3b5-4d1b-bb67-4200f81f0223 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:25.690394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b419dc6-e90f-40da-8402-51a60fe3e5c1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:25.690830 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19795e1-d3b5-4d1b-bb67-4200f81f0223 None admin] PATH_INFO: `/v3/system/users/d5e38a5c2cf44769ab311bdc6af97c92/roles/45c97834bc614c5ca5122d691ce93316` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:25.693485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19795e1-d3b5-4d1b-bb67-4200f81f0223 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d5e38a5c2cf44769ab311bdc6af97c92, role_id=45c97834bc614c5ca5122d691ce93316)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:25.695112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19795e1-d3b5-4d1b-bb67-4200f81f0223 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:25.707688 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 161/332] 10.222.0.46 () {58 vars in 1334 bytes} [Thu May 26 18:03:25 2022] PUT /identity/v3/system/users/d5e38a5c2cf44769ab311bdc6af97c92/roles/45c97834bc614c5ca5122d691ce93316 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:25.709887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:25.739622 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0004b-9f75-4354-bac4-e142b60b9b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:25.740470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0004b-9f75-4354-bac4-e142b60b9b3e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:25.740803 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0004b-9f75-4354-bac4-e142b60b9b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:25.741078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0004b-9f75-4354-bac4-e142b60b9b3e None admin] PATH_INFO: `/v3/users/7dd580e32f474e118d540b28e284682f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:25.742511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0004b-9f75-4354-bac4-e142b60b9b3e None admin] RBAC: Authorizing `identity:get_user(user_id=7dd580e32f474e118d540b28e284682f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:25.743975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0004b-9f75-4354-bac4-e142b60b9b3e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:25.746350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 172/333] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:25 2022] GET /identity/v3/users/7dd580e32f474e118d540b28e284682f => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:25.752996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19795e1-d3b5-4d1b-bb67-4200f81f0223 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:25.805839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:25.900871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19795e1-d3b5-4d1b-bb67-4200f81f0223 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:25.967525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717554e2-2089-4c87-a20e-f61cefdbfc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:25.968198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717554e2-2089-4c87-a20e-f61cefdbfc59 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:25.970457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717554e2-2089-4c87-a20e-f61cefdbfc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:25.970753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717554e2-2089-4c87-a20e-f61cefdbfc59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:25.971610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717554e2-2089-4c87-a20e-f61cefdbfc59 None admin] RBAC: Adding query filter params (role.id=e1a5192e7abc4046bc2c0673176ea1b6, user.id=7dd580e32f474e118d540b28e284682f, scope.system=all) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:25.971948 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717554e2-2089-4c87-a20e-f61cefdbfc59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:25.973463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717554e2-2089-4c87-a20e-f61cefdbfc59 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:25.993087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 162/334] 10.222.0.46 () {56 vars in 1308 bytes} [Thu May 26 18:03:25 2022] GET /identity/v3/role_assignments?user.id=7dd580e32f474e118d540b28e284682f&scope.system=all&role.id=e1a5192e7abc4046bc2c0673176ea1b6 => generated 510 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:28.565235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cbe60-73e6-4351-9582-e05c00ec6ae1 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:28.565796 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cbe60-73e6-4351-9582-e05c00ec6ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:28.566071 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cbe60-73e6-4351-9582-e05c00ec6ae1 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:28.567956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 173/335] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:28.573885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f019d-cab0-46a7-9b4d-3aa1587a0bbf None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:28.575001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f019d-cab0-46a7-9b4d-3aa1587a0bbf None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:28.575645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f019d-cab0-46a7-9b4d-3aa1587a0bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:28.611596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-225f019d-cab0-46a7-9b4d-3aa1587a0bbf None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:28.613381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-225f019d-cab0-46a7-9b4d-3aa1587a0bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:28.709810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 163/336] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:28.754074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cbe60-73e6-4351-9582-e05c00ec6ae1 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:28.905067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cbe60-73e6-4351-9582-e05c00ec6ae1 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:28.920577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7247c-fb58-43d2-ac24-ce03502f7124 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:28.921478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7247c-fb58-43d2-ac24-ce03502f7124 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:28.922562 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7247c-fb58-43d2-ac24-ce03502f7124 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:28.924784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 164/337] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:28.931279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede10611-a115-4840-8f29-15e1db4c8d83 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:28.932208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede10611-a115-4840-8f29-15e1db4c8d83 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:28.932678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede10611-a115-4840-8f29-15e1db4c8d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:28.967974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ede10611-a115-4840-8f29-15e1db4c8d83 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:28.969470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ede10611-a115-4840-8f29-15e1db4c8d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:28.979230 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3faebd-f9d9-400a-9116-8fc2a8c748d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:28.980512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3faebd-f9d9-400a-9116-8fc2a8c748d9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:28.980892 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3faebd-f9d9-400a-9116-8fc2a8c748d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:28.982055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3faebd-f9d9-400a-9116-8fc2a8c748d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:29.021936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3faebd-f9d9-400a-9116-8fc2a8c748d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:29.038387 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-5b3faebd-f9d9-400a-9116-8fc2a8c748d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:29.049201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:29.050175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:29.050936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:29.050936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:29.050936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:29.050936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:29.050936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:29.050936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:29.050936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:29.050936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:29.050936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:29.050936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:29.050936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:29.050936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:29.051532 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 174/338] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:03:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:29.059204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3faebd-f9d9-400a-9116-8fc2a8c748d9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:29.150339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 165/339] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:29.168580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:29.207768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede10611-a115-4840-8f29-15e1db4c8d83 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:29.285400 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3faebd-f9d9-400a-9116-8fc2a8c748d9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:29.374896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f6b11-07a0-45f4-bb25-49cb830af951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:29.375576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f6b11-07a0-45f4-bb25-49cb830af951 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:29.375852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f6b11-07a0-45f4-bb25-49cb830af951 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:29.376118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f6b11-07a0-45f4-bb25-49cb830af951 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:29.376868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f6b11-07a0-45f4-bb25-49cb830af951 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:29.377171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f6b11-07a0-45f4-bb25-49cb830af951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:29.378584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f6b11-07a0-45f4-bb25-49cb830af951 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:29.390165 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 175/340] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:29.391361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bdbb2-5f8c-4aff-bcc5-7bc1f0aa81ca None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:29.391639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bdbb2-5f8c-4aff-bcc5-7bc1f0aa81ca None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:29.391909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bdbb2-5f8c-4aff-bcc5-7bc1f0aa81ca None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:29.393449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 176/341] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:29.399023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bdbb2-5f8c-4aff-bcc5-7bc1f0aa81ca None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:29.436136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede10611-a115-4840-8f29-15e1db4c8d83 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:29.499048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78df31-8f61-446d-b898-e84189300128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:29.500230 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78df31-8f61-446d-b898-e84189300128 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:29.500849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78df31-8f61-446d-b898-e84189300128 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:29.501484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78df31-8f61-446d-b898-e84189300128 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:29.531474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78df31-8f61-446d-b898-e84189300128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-1e78df31-8f61-446d-b898-e84189300128 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:29.544899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:29.545859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:29.546620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:29.546620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:29.546620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:29.546620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:29.546620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:29.546620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:29.546620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:29.546620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:29.546620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:29.546620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:29.546620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:29.546620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:29.547205 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 166/342] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:29.548853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65caff8a-a6b5-4d05-8771-b08bdde756bd None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:29.550060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65caff8a-a6b5-4d05-8771-b08bdde756bd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:29.550060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65caff8a-a6b5-4d05-8771-b08bdde756bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:29.579185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-65caff8a-a6b5-4d05-8771-b08bdde756bd None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:29.582530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-65caff8a-a6b5-4d05-8771-b08bdde756bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:29.603011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:29.604576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bdbb2-5f8c-4aff-bcc5-7bc1f0aa81ca None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:29.610069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:29.652852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bf794-aac6-4ae8-bd0f-040211324844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:29.653999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf794-aac6-4ae8-bd0f-040211324844 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:29.654496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf794-aac6-4ae8-bd0f-040211324844 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:29.654974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf794-aac6-4ae8-bd0f-040211324844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:29.676501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 167/343] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:29.678299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf02a5-e381-42a2-8026-0ab7cd908711 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:29.678805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf02a5-e381-42a2-8026-0ab7cd908711 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:29.679250 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf02a5-e381-42a2-8026-0ab7cd908711 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:29.680737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 168/344] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:29.681720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf02a5-e381-42a2-8026-0ab7cd908711 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:29.745677 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bf794-aac6-4ae8-bd0f-040211324844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:29.748527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bf794-aac6-4ae8-bd0f-040211324844 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:29.806519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf02a5-e381-42a2-8026-0ab7cd908711 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:29.845531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67234ac9-1f92-4528-a7b8-0cecf1da06e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:29.849519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67234ac9-1f92-4528-a7b8-0cecf1da06e6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:29.849519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67234ac9-1f92-4528-a7b8-0cecf1da06e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:29.849519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67234ac9-1f92-4528-a7b8-0cecf1da06e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:29.849519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67234ac9-1f92-4528-a7b8-0cecf1da06e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:29.849519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67234ac9-1f92-4528-a7b8-0cecf1da06e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:29.851843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67234ac9-1f92-4528-a7b8-0cecf1da06e6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:29.865174 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 169/345] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:29.867452 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f388c87-ab51-4691-893f-75cf2a72b7c5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:29.868033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f388c87-ab51-4691-893f-75cf2a72b7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:29.868542 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f388c87-ab51-4691-893f-75cf2a72b7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:29.889143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 177/346] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:29.889143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bf794-aac6-4ae8-bd0f-040211324844 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:29.891265 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:29.894763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3f388c87-ab51-4691-893f-75cf2a72b7c5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:29.896139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3f388c87-ab51-4691-893f-75cf2a72b7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:29.901700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:29.999374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 170/347] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:30.000767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9022a-c17a-4970-b806-24f77ddd25f9 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:30.001067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9022a-c17a-4970-b806-24f77ddd25f9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:30.001338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9022a-c17a-4970-b806-24f77ddd25f9 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:30.002892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 171/348] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:30.003618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9022a-c17a-4970-b806-24f77ddd25f9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:30.016748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bf794-aac6-4ae8-bd0f-040211324844 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:30.061685 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da56bb6-4476-48c2-a1b1-1de5486aa95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:30.062634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da56bb6-4476-48c2-a1b1-1de5486aa95f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:30.062922 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da56bb6-4476-48c2-a1b1-1de5486aa95f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:30.063206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da56bb6-4476-48c2-a1b1-1de5486aa95f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:30.076576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da56bb6-4476-48c2-a1b1-1de5486aa95f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-6da56bb6-4476-48c2-a1b1-1de5486aa95f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:30.085419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:30.086671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:30.087562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 178/349] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:30.088236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da56bb6-4476-48c2-a1b1-1de5486aa95f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:30.174399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9022a-c17a-4970-b806-24f77ddd25f9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:30.178026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:30.218443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116efb96-e013-41d5-99f5-be7d4e28d7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:30.219626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116efb96-e013-41d5-99f5-be7d4e28d7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:30.220193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116efb96-e013-41d5-99f5-be7d4e28d7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:30.220721 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116efb96-e013-41d5-99f5-be7d4e28d7d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:30.235012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da56bb6-4476-48c2-a1b1-1de5486aa95f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:30.235258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:30.284818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5af046-1e9b-4d2d-a9e0-40bada3fab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:30.286784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5af046-1e9b-4d2d-a9e0-40bada3fab0c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:30.286784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5af046-1e9b-4d2d-a9e0-40bada3fab0c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:30.286784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5af046-1e9b-4d2d-a9e0-40bada3fab0c None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:30.290551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5af046-1e9b-4d2d-a9e0-40bada3fab0c None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:30.292307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5af046-1e9b-4d2d-a9e0-40bada3fab0c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:30.297536 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 179/350] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:30 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:30.299021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd5c2d-d121-4208-af22-11949508decf None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:30.299356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd5c2d-d121-4208-af22-11949508decf None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:30.300547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd5c2d-d121-4208-af22-11949508decf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:30.322691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116efb96-e013-41d5-99f5-be7d4e28d7d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:30.323626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-31dd5c2d-d121-4208-af22-11949508decf None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:30.324535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116efb96-e013-41d5-99f5-be7d4e28d7d4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:30.324813 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-31dd5c2d-d121-4208-af22-11949508decf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:30.414573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 180/351] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:30.415355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd5c2d-d121-4208-af22-11949508decf None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:30.454019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 172/352] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:30.457932 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116efb96-e013-41d5-99f5-be7d4e28d7d4 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:30.499361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:30.507427 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:30.560246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd5c2d-d121-4208-af22-11949508decf None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:30.580870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116efb96-e013-41d5-99f5-be7d4e28d7d4 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:30.609562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad1bc5-4af2-45ac-9049-936fafc59efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:30.610394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad1bc5-4af2-45ac-9049-936fafc59efa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:30.610672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad1bc5-4af2-45ac-9049-936fafc59efa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:30.610938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad1bc5-4af2-45ac-9049-936fafc59efa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:30.623577 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad1bc5-4af2-45ac-9049-936fafc59efa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:30.625005 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dabe28-5355-490f-a9b2-4fee2fae284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:30.626117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dabe28-5355-490f-a9b2-4fee2fae284e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:30.626245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dabe28-5355-490f-a9b2-4fee2fae284e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:30.626594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dabe28-5355-490f-a9b2-4fee2fae284e None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:30.627484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dabe28-5355-490f-a9b2-4fee2fae284e None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:30.627843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dabe28-5355-490f-a9b2-4fee2fae284e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:30.629546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dabe28-5355-490f-a9b2-4fee2fae284e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-30ad1bc5-4af2-45ac-9049-936fafc59efa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:30.631687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:30.632506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:30.633284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:30.633284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:30.633284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:30.633284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:30.633284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:30.633284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:30.633284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:30.633284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:30.633284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:30.633284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:30.633284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 18:03:30.633284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:30.634234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 181/353] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:30.635148 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad1bc5-4af2-45ac-9049-936fafc59efa None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:30.643493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 173/354] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:30.645249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172741d7-bd90-4930-bb84-f6e5d5ff8a86 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:30.645605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172741d7-bd90-4930-bb84-f6e5d5ff8a86 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:30.646532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172741d7-bd90-4930-bb84-f6e5d5ff8a86 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:30.648377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 174/355] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:30.649169 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172741d7-bd90-4930-bb84-f6e5d5ff8a86 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:30.771819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad1bc5-4af2-45ac-9049-936fafc59efa None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:30.797809 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172741d7-bd90-4930-bb84-f6e5d5ff8a86 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:30.799473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:30.802212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:30.816497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbc62a-7502-4c35-8736-57153998df9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:30.817274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbc62a-7502-4c35-8736-57153998df9b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:30.817561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbc62a-7502-4c35-8736-57153998df9b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:30.817869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbc62a-7502-4c35-8736-57153998df9b None admin] PATH_INFO: `/v3/groups/086a00dba351454e91ecb6633bf9eaa5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:30.825579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbc62a-7502-4c35-8736-57153998df9b None admin] RBAC: Authorizing `identity:get_group(group_id=086a00dba351454e91ecb6633bf9eaa5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:30.827095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbc62a-7502-4c35-8736-57153998df9b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:30.828838 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 182/356] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:30 2022] GET /identity/v3/groups/086a00dba351454e91ecb6633bf9eaa5 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:30.829626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbc62a-7502-4c35-8736-57153998df9b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:30.836276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291ed55-9888-483b-a9b2-f2e470033a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:30.837136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291ed55-9888-483b-a9b2-f2e470033a54 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:30.837413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291ed55-9888-483b-a9b2-f2e470033a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:30.837701 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291ed55-9888-483b-a9b2-f2e470033a54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:30.851288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291ed55-9888-483b-a9b2-f2e470033a54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-2291ed55-9888-483b-a9b2-f2e470033a54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:30.861485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:30.864417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 175/357] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:03:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:30.866429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291ed55-9888-483b-a9b2-f2e470033a54 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:30.964561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbc62a-7502-4c35-8736-57153998df9b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:30.988947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291ed55-9888-483b-a9b2-f2e470033a54 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:31.007723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951763c-b856-419e-9f47-f9a65910e676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:31.007723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951763c-b856-419e-9f47-f9a65910e676 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:31.008025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951763c-b856-419e-9f47-f9a65910e676 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:31.008235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951763c-b856-419e-9f47-f9a65910e676 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:31.010052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951763c-b856-419e-9f47-f9a65910e676 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:31.011557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951763c-b856-419e-9f47-f9a65910e676 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:31.013345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 183/358] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:30 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:31.014258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951763c-b856-419e-9f47-f9a65910e676 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:31.034617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad95f9c-5f44-47bf-a2c1-d99b8522e9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:31.035600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad95f9c-5f44-47bf-a2c1-d99b8522e9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:31.035882 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad95f9c-5f44-47bf-a2c1-d99b8522e9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:31.036155 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad95f9c-5f44-47bf-a2c1-d99b8522e9e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:31.036951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad95f9c-5f44-47bf-a2c1-d99b8522e9e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:31.037260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad95f9c-5f44-47bf-a2c1-d99b8522e9e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:31.038711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad95f9c-5f44-47bf-a2c1-d99b8522e9e1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:31.051112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 176/359] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:31.052487 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d2f8b-3c32-471a-aa82-dae9541de420 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:31.052779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d2f8b-3c32-471a-aa82-dae9541de420 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:31.053050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d2f8b-3c32-471a-aa82-dae9541de420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:31.071796 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1b8d2f8b-3c32-471a-aa82-dae9541de420 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:31.073883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1b8d2f8b-3c32-471a-aa82-dae9541de420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:31.083154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:31.101360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:31.142139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951763c-b856-419e-9f47-f9a65910e676 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:31.178793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 177/360] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:31.181034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d2f8b-3c32-471a-aa82-dae9541de420 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:31.187772 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e2852-efac-4746-9b2e-ee4fe291f67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:31.188435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e2852-efac-4746-9b2e-ee4fe291f67a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:31.188719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e2852-efac-4746-9b2e-ee4fe291f67a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:31.189085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e2852-efac-4746-9b2e-ee4fe291f67a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:31.273157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e2852-efac-4746-9b2e-ee4fe291f67a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:31.274664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e2852-efac-4746-9b2e-ee4fe291f67a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:31.330254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d2f8b-3c32-471a-aa82-dae9541de420 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:31.353162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:31.376341 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c942930-9b5b-444f-89e6-7eb60418e5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:31.377115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942930-9b5b-444f-89e6-7eb60418e5ab None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:31.377497 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942930-9b5b-444f-89e6-7eb60418e5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:31.380425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942930-9b5b-444f-89e6-7eb60418e5ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:31.380425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c942930-9b5b-444f-89e6-7eb60418e5ab None admin] RBAC: Adding query filter params (group.id=086a00dba351454e91ecb6633bf9eaa5, role.id=506b3cf33324478abe0f72a9d408aa54, scope.project.id=f5b2e96e553d496d96e8f916fb522477) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:31.380425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c942930-9b5b-444f-89e6-7eb60418e5ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:31.381580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c942930-9b5b-444f-89e6-7eb60418e5ab None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:31.391711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 178/361] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 18:03:31 2022] GET /identity/v3/role_assignments?group.id=086a00dba351454e91ecb6633bf9eaa5&scope.project.id=f5b2e96e553d496d96e8f916fb522477&role.id=506b3cf33324478abe0f72a9d408aa54 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:31.392797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c942930-9b5b-444f-89e6-7eb60418e5ab None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:31.420649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 184/362] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:31.425641 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e2852-efac-4746-9b2e-ee4fe291f67a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:31.425782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:31.547975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e2852-efac-4746-9b2e-ee4fe291f67a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:31.558942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c942930-9b5b-444f-89e6-7eb60418e5ab None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:31.587572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fce29-3821-4db1-93dc-7c71097bf027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:31.588559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fce29-3821-4db1-93dc-7c71097bf027 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:31.588995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fce29-3821-4db1-93dc-7c71097bf027 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:31.589272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fce29-3821-4db1-93dc-7c71097bf027 None admin] PATH_INFO: `/v3/groups/086a00dba351454e91ecb6633bf9eaa5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:31.590663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fce29-3821-4db1-93dc-7c71097bf027 None admin] RBAC: Authorizing `identity:get_group(group_id=086a00dba351454e91ecb6633bf9eaa5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:31.593547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fce29-3821-4db1-93dc-7c71097bf027 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:31.595395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 185/363] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:31 2022] GET /identity/v3/groups/086a00dba351454e91ecb6633bf9eaa5 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:31.596249 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fce29-3821-4db1-93dc-7c71097bf027 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:31.602548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479aad5-2b62-4d11-82ea-b3611c0e30ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:31.604063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479aad5-2b62-4d11-82ea-b3611c0e30ec None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:31.604346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479aad5-2b62-4d11-82ea-b3611c0e30ec None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:31.605401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479aad5-2b62-4d11-82ea-b3611c0e30ec None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:31.605504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479aad5-2b62-4d11-82ea-b3611c0e30ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:31.605794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479aad5-2b62-4d11-82ea-b3611c0e30ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:31.608403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479aad5-2b62-4d11-82ea-b3611c0e30ec None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:31.619295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 179/364] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:31.620143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479aad5-2b62-4d11-82ea-b3611c0e30ec None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:31.633434 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:31.719195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:31.727170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fce29-3821-4db1-93dc-7c71097bf027 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:31.741371 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479aad5-2b62-4d11-82ea-b3611c0e30ec None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:31.769642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215cd58-5f43-46a3-b1e0-6bee25f55287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:31.770558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215cd58-5f43-46a3-b1e0-6bee25f55287 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:31.770838 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215cd58-5f43-46a3-b1e0-6bee25f55287 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:31.771108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215cd58-5f43-46a3-b1e0-6bee25f55287 None admin] PATH_INFO: `/v3/users/d5e38a5c2cf44769ab311bdc6af97c92` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:31.772313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215cd58-5f43-46a3-b1e0-6bee25f55287 None admin] RBAC: Authorizing `identity:get_user(user_id=d5e38a5c2cf44769ab311bdc6af97c92)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:31.773800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215cd58-5f43-46a3-b1e0-6bee25f55287 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:31.775480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 186/365] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:03:31 2022] GET /identity/v3/users/d5e38a5c2cf44769ab311bdc6af97c92 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:31.776217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215cd58-5f43-46a3-b1e0-6bee25f55287 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:31.791587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb75994-c117-40c7-84b1-d8a7af6272e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:31.792911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb75994-c117-40c7-84b1-d8a7af6272e7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:31.793345 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb75994-c117-40c7-84b1-d8a7af6272e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:31.793883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb75994-c117-40c7-84b1-d8a7af6272e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:31.805901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb75994-c117-40c7-84b1-d8a7af6272e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-dbb75994-c117-40c7-84b1-d8a7af6272e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:31.814944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:31.816027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 180/366] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:03:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:31.817060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb75994-c117-40c7-84b1-d8a7af6272e7 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:31.883262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:31.894808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215cd58-5f43-46a3-b1e0-6bee25f55287 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:31.934963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb75994-c117-40c7-84b1-d8a7af6272e7 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:31.939485 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5cc03-d6e0-4701-8e7f-a978b534b9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:31.940462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5cc03-d6e0-4701-8e7f-a978b534b9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:31.940765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5cc03-d6e0-4701-8e7f-a978b534b9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:31.941115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5cc03-d6e0-4701-8e7f-a978b534b9f6 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:31.942948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5cc03-d6e0-4701-8e7f-a978b534b9f6 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:31.945033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5cc03-d6e0-4701-8e7f-a978b534b9f6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:31.952897 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 187/367] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:31 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:31.952897 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5cc03-d6e0-4701-8e7f-a978b534b9f6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:31.974743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a19d9c-c364-4f51-9322-cfb75e325ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:31.975934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a19d9c-c364-4f51-9322-cfb75e325ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:31.976271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a19d9c-c364-4f51-9322-cfb75e325ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:31.976663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a19d9c-c364-4f51-9322-cfb75e325ee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:31.977849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a19d9c-c364-4f51-9322-cfb75e325ee5 None admin] RBAC: Adding query filter params (group.id=086a00dba351454e91ecb6633bf9eaa5, role.id=e1a5192e7abc4046bc2c0673176ea1b6, scope.project.id=fea9cdff50914413b637910a718a6cd5) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:31.978551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a19d9c-c364-4f51-9322-cfb75e325ee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:31.980323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a19d9c-c364-4f51-9322-cfb75e325ee5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:31.987829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 181/368] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 18:03:31 2022] GET /identity/v3/role_assignments?group.id=086a00dba351454e91ecb6633bf9eaa5&scope.project.id=fea9cdff50914413b637910a718a6cd5&role.id=e1a5192e7abc4046bc2c0673176ea1b6 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:31.988778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a19d9c-c364-4f51-9322-cfb75e325ee5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:32.006195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:32.111505 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5cc03-d6e0-4701-8e7f-a978b534b9f6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:32.127859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a19d9c-c364-4f51-9322-cfb75e325ee5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:32.151660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc3b19-d787-4140-ab52-d76720dbcda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:32.152822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc3b19-d787-4140-ab52-d76720dbcda1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:32.153864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc3b19-d787-4140-ab52-d76720dbcda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:32.154404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc3b19-d787-4140-ab52-d76720dbcda1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:32.180742 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a733e-19c3-4ccc-9cd7-cb7e37caff92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:32.181461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a733e-19c3-4ccc-9cd7-cb7e37caff92 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:32.181773 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a733e-19c3-4ccc-9cd7-cb7e37caff92 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:32.182056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a733e-19c3-4ccc-9cd7-cb7e37caff92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:32.182890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a733e-19c3-4ccc-9cd7-cb7e37caff92 None admin] RBAC: Adding query filter params (role.id=45c97834bc614c5ca5122d691ce93316, user.id=d5e38a5c2cf44769ab311bdc6af97c92, scope.system=all) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:32.183203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a733e-19c3-4ccc-9cd7-cb7e37caff92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:32.185370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a733e-19c3-4ccc-9cd7-cb7e37caff92 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:32.197693 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:32.211469 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 182/369] 10.222.0.46 () {56 vars in 1308 bytes} [Thu May 26 18:03:31 2022] GET /identity/v3/role_assignments?user.id=d5e38a5c2cf44769ab311bdc6af97c92&scope.system=all&role.id=45c97834bc614c5ca5122d691ce93316 => generated 510 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:32.211469 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a733e-19c3-4ccc-9cd7-cb7e37caff92 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:32.255961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc3b19-d787-4140-ab52-d76720dbcda1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:32.257871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc3b19-d787-4140-ab52-d76720dbcda1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:32.287236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:32.330071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a733e-19c3-4ccc-9cd7-cb7e37caff92 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:32.372877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a4d07-ffdd-4eba-a796-2453854ca7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:32.373544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a4d07-ffdd-4eba-a796-2453854ca7fb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:32.373856 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a4d07-ffdd-4eba-a796-2453854ca7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:32.374133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a4d07-ffdd-4eba-a796-2453854ca7fb None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:32.374885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a4d07-ffdd-4eba-a796-2453854ca7fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:32.375193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a4d07-ffdd-4eba-a796-2453854ca7fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:32.376747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a4d07-ffdd-4eba-a796-2453854ca7fb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:32.385954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 183/370] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:32.391871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a4d07-ffdd-4eba-a796-2453854ca7fb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:32.507471 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a4d07-ffdd-4eba-a796-2453854ca7fb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:32.534451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:32.539782 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 188/371] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:32.540647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc3b19-d787-4140-ab52-d76720dbcda1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:32.548216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda3a69-00a4-41b2-84b2-b4951bfdf5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:32.549078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda3a69-00a4-41b2-84b2-b4951bfdf5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:32.549355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda3a69-00a4-41b2-84b2-b4951bfdf5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:32.549627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda3a69-00a4-41b2-84b2-b4951bfdf5d0 None admin] PATH_INFO: `/v3/groups/086a00dba351454e91ecb6633bf9eaa5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:32.550887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda3a69-00a4-41b2-84b2-b4951bfdf5d0 None admin] RBAC: Authorizing `identity:get_group(group_id=086a00dba351454e91ecb6633bf9eaa5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:32.552348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda3a69-00a4-41b2-84b2-b4951bfdf5d0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:32.554187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 184/372] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:32 2022] GET /identity/v3/groups/086a00dba351454e91ecb6633bf9eaa5 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:32.554882 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda3a69-00a4-41b2-84b2-b4951bfdf5d0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:32.626155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:32.673494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc3b19-d787-4140-ab52-d76720dbcda1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:32.681643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda3a69-00a4-41b2-84b2-b4951bfdf5d0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:32.712826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7c3ac-3204-4473-8731-1e2fff13601a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:32.713953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7c3ac-3204-4473-8731-1e2fff13601a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:32.714437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7c3ac-3204-4473-8731-1e2fff13601a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:32.714926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7c3ac-3204-4473-8731-1e2fff13601a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:32.720639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67749f1-7d67-4190-98c7-7a53fa8acd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:32.721053 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67749f1-7d67-4190-98c7-7a53fa8acd88 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:32.721332 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67749f1-7d67-4190-98c7-7a53fa8acd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:32.721602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67749f1-7d67-4190-98c7-7a53fa8acd88 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:32.722862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67749f1-7d67-4190-98c7-7a53fa8acd88 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:32.724352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67749f1-7d67-4190-98c7-7a53fa8acd88 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:32.727020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 185/373] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:32 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:32.727405 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67749f1-7d67-4190-98c7-7a53fa8acd88 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:32.792730 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:32.799395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff7c3ac-3204-4473-8731-1e2fff13601a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:32.801174 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff7c3ac-3204-4473-8731-1e2fff13601a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:32.842480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67749f1-7d67-4190-98c7-7a53fa8acd88 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:32.878128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:32.880307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce24a83-cc1e-4b96-a1cc-dac0dbc66b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:32.880965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce24a83-cc1e-4b96-a1cc-dac0dbc66b4b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:32.881376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce24a83-cc1e-4b96-a1cc-dac0dbc66b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:32.881655 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce24a83-cc1e-4b96-a1cc-dac0dbc66b4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:32.882787 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce24a83-cc1e-4b96-a1cc-dac0dbc66b4b None admin] RBAC: Adding query filter params (group.id=086a00dba351454e91ecb6633bf9eaa5, role.id=e1a5192e7abc4046bc2c0673176ea1b6, scope.project.id=f5b2e96e553d496d96e8f916fb522477) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:32.883102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce24a83-cc1e-4b96-a1cc-dac0dbc66b4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:32.884736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce24a83-cc1e-4b96-a1cc-dac0dbc66b4b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:32.891154 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 186/374] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 18:03:32 2022] GET /identity/v3/role_assignments?group.id=086a00dba351454e91ecb6633bf9eaa5&scope.project.id=f5b2e96e553d496d96e8f916fb522477&role.id=e1a5192e7abc4046bc2c0673176ea1b6 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:32.891976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce24a83-cc1e-4b96-a1cc-dac0dbc66b4b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:32.936870 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 189/375] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:32.942540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7c3ac-3204-4473-8731-1e2fff13601a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:33.040589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce24a83-cc1e-4b96-a1cc-dac0dbc66b4b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:33.064312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7c3ac-3204-4473-8731-1e2fff13601a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:33.072154 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:33.091424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1913aba-79e9-441f-b990-f76d800b90a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:33.092289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1913aba-79e9-441f-b990-f76d800b90a2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:33.092568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1913aba-79e9-441f-b990-f76d800b90a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:33.092836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1913aba-79e9-441f-b990-f76d800b90a2 None admin] PATH_INFO: `/v3/groups/086a00dba351454e91ecb6633bf9eaa5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:33.094099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1913aba-79e9-441f-b990-f76d800b90a2 None admin] RBAC: Authorizing `identity:get_group(group_id=086a00dba351454e91ecb6633bf9eaa5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:33.095767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1913aba-79e9-441f-b990-f76d800b90a2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:33.097395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 187/376] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:32 2022] GET /identity/v3/groups/086a00dba351454e91ecb6633bf9eaa5 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:33.104446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1913aba-79e9-441f-b990-f76d800b90a2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:33.107655 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264439c-1674-43b0-a2bc-ee0ed9627e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:33.108726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264439c-1674-43b0-a2bc-ee0ed9627e69 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:33.109001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264439c-1674-43b0-a2bc-ee0ed9627e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:33.109272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264439c-1674-43b0-a2bc-ee0ed9627e69 None admin] PATH_INFO: `/v3/projects/bbbdb1cacacf4ba5a41a00d6d478efbe` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:33.110598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264439c-1674-43b0-a2bc-ee0ed9627e69 None admin] RBAC: Authorizing `identity:get_project(project_id=bbbdb1cacacf4ba5a41a00d6d478efbe)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:33.112082 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264439c-1674-43b0-a2bc-ee0ed9627e69 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:33.114206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 190/377] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:32 2022] GET /identity/v3/projects/bbbdb1cacacf4ba5a41a00d6d478efbe => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:33.119657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264439c-1674-43b0-a2bc-ee0ed9627e69 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:33.152667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:33.236316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264439c-1674-43b0-a2bc-ee0ed9627e69 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:33.282711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce999f-9cdd-4cf4-85e3-17c1c924b4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:33.283674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce999f-9cdd-4cf4-85e3-17c1c924b4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:33.283951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce999f-9cdd-4cf4-85e3-17c1c924b4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:33.284222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce999f-9cdd-4cf4-85e3-17c1c924b4b4 None admin] PATH_INFO: `/v3/groups/ee8d6a358ca94bb798b47effa5c74506` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:33.291890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce999f-9cdd-4cf4-85e3-17c1c924b4b4 None admin] RBAC: Authorizing `identity:get_group(group_id=ee8d6a358ca94bb798b47effa5c74506)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:33.293340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce999f-9cdd-4cf4-85e3-17c1c924b4b4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:33.295165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1913aba-79e9-441f-b990-f76d800b90a2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:33.295767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 191/378] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:33 2022] GET /identity/v3/groups/ee8d6a358ca94bb798b47effa5c74506 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:33.302234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce999f-9cdd-4cf4-85e3-17c1c924b4b4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:33.346020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a120ee-3f9e-4c9c-9c91-7723c8c8b2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:33.347350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a120ee-3f9e-4c9c-9c91-7723c8c8b2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:33.347755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a120ee-3f9e-4c9c-9c91-7723c8c8b2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:33.348029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a120ee-3f9e-4c9c-9c91-7723c8c8b2f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:33.348935 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a120ee-3f9e-4c9c-9c91-7723c8c8b2f9 None admin] RBAC: Adding query filter params (group.id=086a00dba351454e91ecb6633bf9eaa5, role.id=506b3cf33324478abe0f72a9d408aa54, scope.project.id=fea9cdff50914413b637910a718a6cd5) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:33.349286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a120ee-3f9e-4c9c-9c91-7723c8c8b2f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:33.351115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a120ee-3f9e-4c9c-9c91-7723c8c8b2f9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:33.359587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 188/379] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 18:03:33 2022] GET /identity/v3/role_assignments?group.id=086a00dba351454e91ecb6633bf9eaa5&scope.project.id=fea9cdff50914413b637910a718a6cd5&role.id=506b3cf33324478abe0f72a9d408aa54 => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:33.425826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:33.439844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce999f-9cdd-4cf4-85e3-17c1c924b4b4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:33.483450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1e7a7-f368-4d15-b03c-35a5ced354d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:33.484206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1e7a7-f368-4d15-b03c-35a5ced354d0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:33.484558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1e7a7-f368-4d15-b03c-35a5ced354d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:33.484882 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1e7a7-f368-4d15-b03c-35a5ced354d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:33.485749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1e7a7-f368-4d15-b03c-35a5ced354d0 None admin] RBAC: Adding query filter params (group.id=ee8d6a358ca94bb798b47effa5c74506, role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=bbbdb1cacacf4ba5a41a00d6d478efbe) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:33.486059 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1e7a7-f368-4d15-b03c-35a5ced354d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:33.487542 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1e7a7-f368-4d15-b03c-35a5ced354d0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:33.494679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 192/380] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 18:03:33 2022] GET /identity/v3/role_assignments?group.id=ee8d6a358ca94bb798b47effa5c74506&scope.project.id=bbbdb1cacacf4ba5a41a00d6d478efbe&role.id=584beecb022b422f820e77f7623ada5e => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:34.203591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64420f-24f9-4745-b5c6-089bbd586f8c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:34.204290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64420f-24f9-4745-b5c6-089bbd586f8c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:34.204290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64420f-24f9-4745-b5c6-089bbd586f8c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:34.205808 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 189/381] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:34.211241 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a00d9b-d4e5-42cf-9d5d-bd1c5844aebf None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:34.211790 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a00d9b-d4e5-42cf-9d5d-bd1c5844aebf None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:34.211790 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a00d9b-d4e5-42cf-9d5d-bd1c5844aebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:34.238709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f0a00d9b-d4e5-42cf-9d5d-bd1c5844aebf None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:34.239934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f0a00d9b-d4e5-42cf-9d5d-bd1c5844aebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:34.343171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 193/382] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:34.378092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64420f-24f9-4745-b5c6-089bbd586f8c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:34.467481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:34.531820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64420f-24f9-4745-b5c6-089bbd586f8c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:34.583802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89244a-4791-405b-975d-6deb24d4565c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:34.584970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89244a-4791-405b-975d-6deb24d4565c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:34.585472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89244a-4791-405b-975d-6deb24d4565c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:34.585971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89244a-4791-405b-975d-6deb24d4565c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:34.605777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89244a-4791-405b-975d-6deb24d4565c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8b89244a-4791-405b-975d-6deb24d4565c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:34.614530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:34.615662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:34.616952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 190/383] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:03:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:34.622381 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a00d9b-d4e5-42cf-9d5d-bd1c5844aebf None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:34.728449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a5ef5-a227-4333-92d4-74ca3feef297 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:34.729163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a5ef5-a227-4333-92d4-74ca3feef297 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:34.729637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a5ef5-a227-4333-92d4-74ca3feef297 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:34.731512 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 191/384] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:34.737167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26e1fc-9718-4590-860b-f9723e8376a8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:34.737848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26e1fc-9718-4590-860b-f9723e8376a8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:34.738340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26e1fc-9718-4590-860b-f9723e8376a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:34.755990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:34.770411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3c26e1fc-9718-4590-860b-f9723e8376a8 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:34.773966 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3c26e1fc-9718-4590-860b-f9723e8376a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:34.789005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a00d9b-d4e5-42cf-9d5d-bd1c5844aebf None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:34.866069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae8eca-98ac-430f-8292-2712c00fcf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:34.867101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae8eca-98ac-430f-8292-2712c00fcf85 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:34.867576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae8eca-98ac-430f-8292-2712c00fcf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:34.868065 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae8eca-98ac-430f-8292-2712c00fcf85 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:34.869044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae8eca-98ac-430f-8292-2712c00fcf85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:34.869557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae8eca-98ac-430f-8292-2712c00fcf85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:34.871239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae8eca-98ac-430f-8292-2712c00fcf85 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:34.889702 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 194/385] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:34.895385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae8eca-98ac-430f-8292-2712c00fcf85 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:34.932494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 192/386] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:34.967979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26e1fc-9718-4590-860b-f9723e8376a8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:35.031235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:35.095158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26e1fc-9718-4590-860b-f9723e8376a8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:35.109749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae8eca-98ac-430f-8292-2712c00fcf85 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:35.146539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5c1ea-5353-4876-8437-8820beb56025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:35.147973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5c1ea-5353-4876-8437-8820beb56025 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:35.148556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5c1ea-5353-4876-8437-8820beb56025 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:35.149144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5c1ea-5353-4876-8437-8820beb56025 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:35.167653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a63f5f-ce5d-4d0b-afef-006fcd936696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:35.168569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a63f5f-ce5d-4d0b-afef-006fcd936696 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:35.169521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a63f5f-ce5d-4d0b-afef-006fcd936696 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:35.170063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a63f5f-ce5d-4d0b-afef-006fcd936696 None admin] PATH_INFO: `/v3/groups/086a00dba351454e91ecb6633bf9eaa5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:35.171625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5c1ea-5353-4876-8437-8820beb56025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-b1b5c1ea-5353-4876-8437-8820beb56025 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:35.180363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:35.181224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:35.182015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 193/387] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:35.182747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a63f5f-ce5d-4d0b-afef-006fcd936696 None admin] RBAC: Authorizing `identity:get_group(group_id=086a00dba351454e91ecb6633bf9eaa5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:35.184058 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a63f5f-ce5d-4d0b-afef-006fcd936696 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:35.191811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 195/388] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:34 2022] GET /identity/v3/groups/086a00dba351454e91ecb6633bf9eaa5 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:35.192801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5c1ea-5353-4876-8437-8820beb56025 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:35.197197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a63f5f-ce5d-4d0b-afef-006fcd936696 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:35.226511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:35.408200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5c1ea-5353-4876-8437-8820beb56025 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:35.409036 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a63f5f-ce5d-4d0b-afef-006fcd936696 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:35.415878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:35.461626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d46f0-64c1-4788-940c-ef0cd9dbedc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:35.462953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d46f0-64c1-4788-940c-ef0cd9dbedc2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:35.463241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d46f0-64c1-4788-940c-ef0cd9dbedc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:35.463489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d46f0-64c1-4788-940c-ef0cd9dbedc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:35.464246 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d46f0-64c1-4788-940c-ef0cd9dbedc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:35.464552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d46f0-64c1-4788-940c-ef0cd9dbedc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:35.466705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d46f0-64c1-4788-940c-ef0cd9dbedc2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:35.471991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e27142-26ca-4ee3-bb55-4bba96936ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:35.472854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e27142-26ca-4ee3-bb55-4bba96936ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:35.473138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e27142-26ca-4ee3-bb55-4bba96936ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:35.473371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e27142-26ca-4ee3-bb55-4bba96936ae3 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:35.475101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e27142-26ca-4ee3-bb55-4bba96936ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:35.476687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e27142-26ca-4ee3-bb55-4bba96936ae3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:35.478679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 196/389] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:35 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:35.489649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 194/390] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:35.493922 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e27142-26ca-4ee3-bb55-4bba96936ae3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:35.496354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d46f0-64c1-4788-940c-ef0cd9dbedc2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:35.611757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:35.647813 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e27142-26ca-4ee3-bb55-4bba96936ae3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:35.673085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d46f0-64c1-4788-940c-ef0cd9dbedc2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:35.710661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911ce80-23fe-4bc7-8a3f-db2747191937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:35.711605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911ce80-23fe-4bc7-8a3f-db2747191937 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:35.711951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911ce80-23fe-4bc7-8a3f-db2747191937 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:35.712378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911ce80-23fe-4bc7-8a3f-db2747191937 None admin] PATH_INFO: `/v3/groups/086a00dba351454e91ecb6633bf9eaa5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:35.714430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911ce80-23fe-4bc7-8a3f-db2747191937 None admin] RBAC: Authorizing `identity:get_group(group_id=086a00dba351454e91ecb6633bf9eaa5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:35.716148 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911ce80-23fe-4bc7-8a3f-db2747191937 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:35.718642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 195/391] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:35 2022] GET /identity/v3/groups/086a00dba351454e91ecb6633bf9eaa5 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:35.724437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911ce80-23fe-4bc7-8a3f-db2747191937 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:35.780124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a21a6-d92d-4ff7-bcea-4df5dc58e3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:35.780741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:35.781053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a21a6-d92d-4ff7-bcea-4df5dc58e3aa None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:35.781335 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a21a6-d92d-4ff7-bcea-4df5dc58e3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:35.781609 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a21a6-d92d-4ff7-bcea-4df5dc58e3aa None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477/groups/086a00dba351454e91ecb6633bf9eaa5/roles/506b3cf33324478abe0f72a9d408aa54` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:35.798604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a21a6-d92d-4ff7-bcea-4df5dc58e3aa None admin] RBAC: Authorizing `identity:create_grant(project_id=f5b2e96e553d496d96e8f916fb522477, group_id=086a00dba351454e91ecb6633bf9eaa5, role_id=506b3cf33324478abe0f72a9d408aa54)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:35.800068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a21a6-d92d-4ff7-bcea-4df5dc58e3aa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:35.835046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 197/392] 10.222.0.46 () {58 vars in 1442 bytes} [Thu May 26 18:03:35 2022] PUT /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477/groups/086a00dba351454e91ecb6633bf9eaa5/roles/506b3cf33324478abe0f72a9d408aa54 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:35.838092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911ce80-23fe-4bc7-8a3f-db2747191937 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:35.906670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def25474-dd2a-4215-97b3-789684e02e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:35.907555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def25474-dd2a-4215-97b3-789684e02e89 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:35.907832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def25474-dd2a-4215-97b3-789684e02e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:35.908101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def25474-dd2a-4215-97b3-789684e02e89 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:35.911098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def25474-dd2a-4215-97b3-789684e02e89 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:35.912688 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def25474-dd2a-4215-97b3-789684e02e89 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:35.914851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 196/393] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:35 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:35.922874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a21a6-d92d-4ff7-bcea-4df5dc58e3aa None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:36.063930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a21a6-d92d-4ff7-bcea-4df5dc58e3aa None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:36.080140 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf88ed-c796-4ca8-a301-0d3450d58258 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:36.080493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf88ed-c796-4ca8-a301-0d3450d58258 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:36.080682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf88ed-c796-4ca8-a301-0d3450d58258 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:36.083420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 197/394] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:03:36.091792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496eb0e-cfee-4dac-bf70-8ddb7a230007 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:36.092414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496eb0e-cfee-4dac-bf70-8ddb7a230007 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:36.092912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496eb0e-cfee-4dac-bf70-8ddb7a230007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:36.109107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb299f-f6f8-4ef6-9ebd-a5a025a3061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:36.110100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb299f-f6f8-4ef6-9ebd-a5a025a3061e None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:36.110453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb299f-f6f8-4ef6-9ebd-a5a025a3061e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:36.110734 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb299f-f6f8-4ef6-9ebd-a5a025a3061e None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5/groups/086a00dba351454e91ecb6633bf9eaa5/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:36.112899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb299f-f6f8-4ef6-9ebd-a5a025a3061e None admin] RBAC: Authorizing `identity:create_grant(project_id=fea9cdff50914413b637910a718a6cd5, group_id=086a00dba351454e91ecb6633bf9eaa5, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:36.114581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb299f-f6f8-4ef6-9ebd-a5a025a3061e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:36.115310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4496eb0e-cfee-4dac-bf70-8ddb7a230007 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:36.116786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4496eb0e-cfee-4dac-bf70-8ddb7a230007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:36.134877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 198/395] 10.222.0.46 () {58 vars in 1442 bytes} [Thu May 26 18:03:35 2022] PUT /identity/v3/projects/fea9cdff50914413b637910a718a6cd5/groups/086a00dba351454e91ecb6633bf9eaa5/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:36.235168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 198/396] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:36.273948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb299f-f6f8-4ef6-9ebd-a5a025a3061e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:36.277271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:36.312281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb822f18-ef6b-4cf3-bab5-ecc349485ca0 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:36.312972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb822f18-ef6b-4cf3-bab5-ecc349485ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:36.313487 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb822f18-ef6b-4cf3-bab5-ecc349485ca0 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:36.321482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 199/397] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:03:36.328969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db54c3-737e-4b6f-8fc4-453c12f75517 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:36.329617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db54c3-737e-4b6f-8fc4-453c12f75517 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:36.330207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db54c3-737e-4b6f-8fc4-453c12f75517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:36.355969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-87db54c3-737e-4b6f-8fc4-453c12f75517 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:36.357851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-87db54c3-737e-4b6f-8fc4-453c12f75517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:36.397823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:36.423453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb299f-f6f8-4ef6-9ebd-a5a025a3061e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:36.460618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 200/398] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:36.463937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3afa07-e322-4b12-80d7-3fcad30dbe24 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:36.464237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3afa07-e322-4b12-80d7-3fcad30dbe24 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:36.465093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3afa07-e322-4b12-80d7-3fcad30dbe24 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:36.468588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 201/399] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:36.475460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcfff3-52a2-469e-8209-461e5f5d6745 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:36.475842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcfff3-52a2-469e-8209-461e5f5d6745 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:36.476126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcfff3-52a2-469e-8209-461e5f5d6745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:36.479627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753908c1-1712-41f7-959b-158522a1fa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:36.481058 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753908c1-1712-41f7-959b-158522a1fa43 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:36.481579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753908c1-1712-41f7-959b-158522a1fa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:36.481987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753908c1-1712-41f7-959b-158522a1fa43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:36.500089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-80fcfff3-52a2-469e-8209-461e5f5d6745 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:36.501031 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753908c1-1712-41f7-959b-158522a1fa43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:36.501185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-80fcfff3-52a2-469e-8209-461e5f5d6745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-753908c1-1712-41f7-959b-158522a1fa43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:36.511667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:36.513307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:36.514337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:36.514337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:36.514337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:36.514337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:36.514337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:36.514337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:36.514337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:36.514337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:36.514337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:36.514337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:36.514337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:36.514337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:36.515006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 199/400] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:36.519959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753908c1-1712-41f7-959b-158522a1fa43 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:36.604975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:36.609989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 202/401] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:36.611233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcfff3-52a2-469e-8209-461e5f5d6745 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:36.658417 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753908c1-1712-41f7-959b-158522a1fa43 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:36.694557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba13556-757d-4d01-8e25-b3d66e3e35d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:36.695732 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba13556-757d-4d01-8e25-b3d66e3e35d0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:36.696205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba13556-757d-4d01-8e25-b3d66e3e35d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:36.696686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba13556-757d-4d01-8e25-b3d66e3e35d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:36.708990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba13556-757d-4d01-8e25-b3d66e3e35d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-0ba13556-757d-4d01-8e25-b3d66e3e35d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:36.718018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:36.718951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:36.719728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:36.719728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:36.719728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:36.719728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:36.719728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:36.719728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:36.719728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:36.719728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:36.719728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:36.719728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:36.719728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:36.719728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:36.722531 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 200/402] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:03:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:36.723692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba13556-757d-4d01-8e25-b3d66e3e35d0 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:36.738075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:36.749940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcfff3-52a2-469e-8209-461e5f5d6745 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:36.804375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cab352-5f17-425e-b1ef-cade168ff25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:36.805057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cab352-5f17-425e-b1ef-cade168ff25e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:36.805444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cab352-5f17-425e-b1ef-cade168ff25e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:36.805754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cab352-5f17-425e-b1ef-cade168ff25e None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:36.806776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cab352-5f17-425e-b1ef-cade168ff25e None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:36.807086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cab352-5f17-425e-b1ef-cade168ff25e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:36.809025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cab352-5f17-425e-b1ef-cade168ff25e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:36.823156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 203/403] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:36.824371 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cab352-5f17-425e-b1ef-cade168ff25e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:36.847867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba13556-757d-4d01-8e25-b3d66e3e35d0 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:36.879545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:36.900588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b25b7c-ce35-4547-bcee-e7badb75dbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:36.901477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b25b7c-ce35-4547-bcee-e7badb75dbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:36.902200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b25b7c-ce35-4547-bcee-e7badb75dbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:36.902493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b25b7c-ce35-4547-bcee-e7badb75dbb4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:36.919456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b25b7c-ce35-4547-bcee-e7badb75dbb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-22b25b7c-ce35-4547-bcee-e7badb75dbb4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:36.929864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:36.930791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 201/404] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:03:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:36.938227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b25b7c-ce35-4547-bcee-e7badb75dbb4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:36.974969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cab352-5f17-425e-b1ef-cade168ff25e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:37.022963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7491c-25fc-4aa9-8b94-76d93285842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:37.023639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7491c-25fc-4aa9-8b94-76d93285842d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:37.023921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7491c-25fc-4aa9-8b94-76d93285842d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:37.024189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7491c-25fc-4aa9-8b94-76d93285842d None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:37.024937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7491c-25fc-4aa9-8b94-76d93285842d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:37.025237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7491c-25fc-4aa9-8b94-76d93285842d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:37.026701 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7491c-25fc-4aa9-8b94-76d93285842d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:37.039026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 204/405] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:37.041199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7491c-25fc-4aa9-8b94-76d93285842d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:37.068259 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:37.077873 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b25b7c-ce35-4547-bcee-e7badb75dbb4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:37.129544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996164f-2bff-455b-b06e-48aa8fb97a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:37.130437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996164f-2bff-455b-b06e-48aa8fb97a57 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:37.130713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996164f-2bff-455b-b06e-48aa8fb97a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:37.130987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996164f-2bff-455b-b06e-48aa8fb97a57 None admin] PATH_INFO: `/v3/groups/086a00dba351454e91ecb6633bf9eaa5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:37.132171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996164f-2bff-455b-b06e-48aa8fb97a57 None admin] RBAC: Authorizing `identity:get_group(group_id=086a00dba351454e91ecb6633bf9eaa5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:37.133622 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996164f-2bff-455b-b06e-48aa8fb97a57 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:37.135872 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 202/406] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:36 2022] GET /identity/v3/groups/086a00dba351454e91ecb6633bf9eaa5 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:37.136662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996164f-2bff-455b-b06e-48aa8fb97a57 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:37.176978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7491c-25fc-4aa9-8b94-76d93285842d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:37.206270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:37.231824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facc66a-f114-4daa-b886-048ea3595011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:37.232729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facc66a-f114-4daa-b886-048ea3595011 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:37.233007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facc66a-f114-4daa-b886-048ea3595011 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:37.233274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facc66a-f114-4daa-b886-048ea3595011 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:37.234056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facc66a-f114-4daa-b886-048ea3595011 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:37.234359 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facc66a-f114-4daa-b886-048ea3595011 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:37.235874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facc66a-f114-4daa-b886-048ea3595011 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:37.252037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 205/407] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:37.253059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facc66a-f114-4daa-b886-048ea3595011 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:37.281405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996164f-2bff-455b-b06e-48aa8fb97a57 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:37.322571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb749b-6f6a-4835-b425-8b3c4f2e9f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:37.324549 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb749b-6f6a-4835-b425-8b3c4f2e9f26 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:37.325235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb749b-6f6a-4835-b425-8b3c4f2e9f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:37.325935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb749b-6f6a-4835-b425-8b3c4f2e9f26 None admin] PATH_INFO: `/v3/groups/ee8d6a358ca94bb798b47effa5c74506` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:37.327442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb749b-6f6a-4835-b425-8b3c4f2e9f26 None admin] RBAC: Authorizing `identity:get_group(group_id=ee8d6a358ca94bb798b47effa5c74506)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:37.329237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb749b-6f6a-4835-b425-8b3c4f2e9f26 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:37.331546 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 203/408] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:37 2022] GET /identity/v3/groups/ee8d6a358ca94bb798b47effa5c74506 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:37.337015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb749b-6f6a-4835-b425-8b3c4f2e9f26 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:37.350134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:37.371880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facc66a-f114-4daa-b886-048ea3595011 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:37.407776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91da3b-ee03-4843-adcf-509b04263bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:37.408637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91da3b-ee03-4843-adcf-509b04263bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:37.408918 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91da3b-ee03-4843-adcf-509b04263bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:37.409181 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91da3b-ee03-4843-adcf-509b04263bb7 None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:37.412205 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91da3b-ee03-4843-adcf-509b04263bb7 None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:37.414802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91da3b-ee03-4843-adcf-509b04263bb7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:37.418257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 206/409] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:37 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:37.420374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91da3b-ee03-4843-adcf-509b04263bb7 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:37.488936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb749b-6f6a-4835-b425-8b3c4f2e9f26 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:37.509003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:37.528011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88a946-b380-4af7-9372-d94bec59c621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:37.529835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88a946-b380-4af7-9372-d94bec59c621 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:37.530263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88a946-b380-4af7-9372-d94bec59c621 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:37.530676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88a946-b380-4af7-9372-d94bec59c621 None admin] PATH_INFO: `/v3/groups/086a00dba351454e91ecb6633bf9eaa5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:37.532082 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88a946-b380-4af7-9372-d94bec59c621 None admin] RBAC: Authorizing `identity:get_group(group_id=086a00dba351454e91ecb6633bf9eaa5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:37.533659 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88a946-b380-4af7-9372-d94bec59c621 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:37.535763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 204/410] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:37 2022] GET /identity/v3/groups/086a00dba351454e91ecb6633bf9eaa5 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:37.536627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88a946-b380-4af7-9372-d94bec59c621 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:37.613181 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91da3b-ee03-4843-adcf-509b04263bb7 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:37.647044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88a946-b380-4af7-9372-d94bec59c621 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:37.681766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2aa14f-a171-421c-9959-0d49c1d32d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:37.682998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2aa14f-a171-421c-9959-0d49c1d32d1d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:37.683277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2aa14f-a171-421c-9959-0d49c1d32d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:37.683554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2aa14f-a171-421c-9959-0d49c1d32d1d None admin] PATH_INFO: `/v3/projects/bbbdb1cacacf4ba5a41a00d6d478efbe` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:37.686333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2aa14f-a171-421c-9959-0d49c1d32d1d None admin] RBAC: Authorizing `identity:get_project(project_id=bbbdb1cacacf4ba5a41a00d6d478efbe)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:37.687814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2aa14f-a171-421c-9959-0d49c1d32d1d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:37.689880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 207/411] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:37 2022] GET /identity/v3/projects/bbbdb1cacacf4ba5a41a00d6d478efbe => generated 344 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:37.690134 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bdf78-0df9-4395-9f27-8529bc31f46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:37.690790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2aa14f-a171-421c-9959-0d49c1d32d1d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:37.691581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bdf78-0df9-4395-9f27-8529bc31f46b None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:37.692070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bdf78-0df9-4395-9f27-8529bc31f46b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:37.692578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bdf78-0df9-4395-9f27-8529bc31f46b None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477/groups/086a00dba351454e91ecb6633bf9eaa5/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:37.694792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bdf78-0df9-4395-9f27-8529bc31f46b None admin] RBAC: Authorizing `identity:create_grant(project_id=f5b2e96e553d496d96e8f916fb522477, group_id=086a00dba351454e91ecb6633bf9eaa5, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:37.696561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bdf78-0df9-4395-9f27-8529bc31f46b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:37.714007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:37.721761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 205/412] 10.222.0.46 () {58 vars in 1442 bytes} [Thu May 26 18:03:37 2022] PUT /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477/groups/086a00dba351454e91ecb6633bf9eaa5/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:37.723225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bdf78-0df9-4395-9f27-8529bc31f46b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:37.812412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2aa14f-a171-421c-9959-0d49c1d32d1d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:37.820460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:37.856173 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d318f62-85c2-4f20-8c5e-daec0bfaa21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:37.858469 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d318f62-85c2-4f20-8c5e-daec0bfaa21d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:37.858949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d318f62-85c2-4f20-8c5e-daec0bfaa21d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:37.859412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d318f62-85c2-4f20-8c5e-daec0bfaa21d None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:37.860866 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d318f62-85c2-4f20-8c5e-daec0bfaa21d None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:37.862589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d318f62-85c2-4f20-8c5e-daec0bfaa21d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:37.869497 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 208/413] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:37 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:37.875707 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d318f62-85c2-4f20-8c5e-daec0bfaa21d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:37.968469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bdf78-0df9-4395-9f27-8529bc31f46b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:37.991462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:38.013574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d318f62-85c2-4f20-8c5e-daec0bfaa21d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:38.043230 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9311ab7-7711-4e46-9ebc-e85d2bf7a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:38.044389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9311ab7-7711-4e46-9ebc-e85d2bf7a9fc None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:38.044756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9311ab7-7711-4e46-9ebc-e85d2bf7a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:38.044981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9311ab7-7711-4e46-9ebc-e85d2bf7a9fc None admin] PATH_INFO: `/v3/projects/bbbdb1cacacf4ba5a41a00d6d478efbe/groups/ee8d6a358ca94bb798b47effa5c74506/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:38.047236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9311ab7-7711-4e46-9ebc-e85d2bf7a9fc None admin] RBAC: Authorizing `identity:create_grant(project_id=bbbdb1cacacf4ba5a41a00d6d478efbe, group_id=ee8d6a358ca94bb798b47effa5c74506, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:38.048875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9311ab7-7711-4e46-9ebc-e85d2bf7a9fc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:38.052795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b955db2-b18a-4263-a6b8-dcdf5270ea19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:38.053970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b955db2-b18a-4263-a6b8-dcdf5270ea19 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:38.054476 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b955db2-b18a-4263-a6b8-dcdf5270ea19 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:38.054948 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b955db2-b18a-4263-a6b8-dcdf5270ea19 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5/groups/086a00dba351454e91ecb6633bf9eaa5/roles/506b3cf33324478abe0f72a9d408aa54` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:38.058119 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b955db2-b18a-4263-a6b8-dcdf5270ea19 None admin] RBAC: Authorizing `identity:create_grant(project_id=fea9cdff50914413b637910a718a6cd5, group_id=086a00dba351454e91ecb6633bf9eaa5, role_id=506b3cf33324478abe0f72a9d408aa54)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:38.060374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b955db2-b18a-4263-a6b8-dcdf5270ea19 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:38.066769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 206/414] 10.222.0.46 () {58 vars in 1442 bytes} [Thu May 26 18:03:37 2022] PUT /identity/v3/projects/bbbdb1cacacf4ba5a41a00d6d478efbe/groups/ee8d6a358ca94bb798b47effa5c74506/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:38.079730 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 209/415] 10.222.0.46 () {58 vars in 1442 bytes} [Thu May 26 18:03:37 2022] PUT /identity/v3/projects/fea9cdff50914413b637910a718a6cd5/groups/086a00dba351454e91ecb6633bf9eaa5/roles/506b3cf33324478abe0f72a9d408aa54 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:38.440048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7810ed-b312-48e2-b711-be4457d41174 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:38.440548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7810ed-b312-48e2-b711-be4457d41174 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:38.440647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7810ed-b312-48e2-b711-be4457d41174 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:38.442304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 207/416] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:38.448130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68501c-d7ea-4a8d-8807-6207cd5bb703 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:38.448412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68501c-d7ea-4a8d-8807-6207cd5bb703 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:38.448748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68501c-d7ea-4a8d-8807-6207cd5bb703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:38.474136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6c68501c-d7ea-4a8d-8807-6207cd5bb703 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:38.475459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6c68501c-d7ea-4a8d-8807-6207cd5bb703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:38.587420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 210/417] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:38.635286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7810ed-b312-48e2-b711-be4457d41174 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:38.767577 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7810ed-b312-48e2-b711-be4457d41174 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:38.775118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:38.816959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bfd07-022f-44b6-8d75-96b9c6e43a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:38.818182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bfd07-022f-44b6-8d75-96b9c6e43a3b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:38.818664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bfd07-022f-44b6-8d75-96b9c6e43a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:38.819307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bfd07-022f-44b6-8d75-96b9c6e43a3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:38.843817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bfd07-022f-44b6-8d75-96b9c6e43a3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:38.852533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebfaf8-61e7-40f6-96a2-0ff25f8063d0 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:38.853003 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebfaf8-61e7-40f6-96a2-0ff25f8063d0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:38.853079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebfaf8-61e7-40f6-96a2-0ff25f8063d0 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:38.854519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 211/418] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-428bfd07-022f-44b6-8d75-96b9c6e43a3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:38.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:38.857015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:38.858246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 208/419] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:03:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:38.863555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716da91e-7b6f-45cb-a7d1-0b6f0b7009cb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:38.863842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716da91e-7b6f-45cb-a7d1-0b6f0b7009cb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:38.864120 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716da91e-7b6f-45cb-a7d1-0b6f0b7009cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:38.865704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bfd07-022f-44b6-8d75-96b9c6e43a3b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:38.899029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-716da91e-7b6f-45cb-a7d1-0b6f0b7009cb None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:38.902052 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-716da91e-7b6f-45cb-a7d1-0b6f0b7009cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:39.078058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 212/420] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:39.090946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bfd07-022f-44b6-8d75-96b9c6e43a3b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:39.110894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716da91e-7b6f-45cb-a7d1-0b6f0b7009cb None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:39.126082 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:39.162483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432b140-4375-402a-b1f4-cf5929322bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:39.163158 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432b140-4375-402a-b1f4-cf5929322bca None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:39.163433 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432b140-4375-402a-b1f4-cf5929322bca None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:39.163700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432b140-4375-402a-b1f4-cf5929322bca None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:39.164436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432b140-4375-402a-b1f4-cf5929322bca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:39.164736 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432b140-4375-402a-b1f4-cf5929322bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:39.166123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432b140-4375-402a-b1f4-cf5929322bca None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:39.181162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 209/421] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:39.210606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432b140-4375-402a-b1f4-cf5929322bca None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:39.265582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:39.302267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716da91e-7b6f-45cb-a7d1-0b6f0b7009cb None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:39.366924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432b140-4375-402a-b1f4-cf5929322bca None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:39.402171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e8a4c-8af3-4631-812e-56027cbb6745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:39.402819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e8a4c-8af3-4631-812e-56027cbb6745 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:39.403172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e8a4c-8af3-4631-812e-56027cbb6745 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:39.403566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e8a4c-8af3-4631-812e-56027cbb6745 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:39.436976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de87d8f-55a3-4946-86a7-acf5217f6c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:39.437875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de87d8f-55a3-4946-86a7-acf5217f6c6b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:39.438157 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de87d8f-55a3-4946-86a7-acf5217f6c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:39.438482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de87d8f-55a3-4946-86a7-acf5217f6c6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:39.465872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de87d8f-55a3-4946-86a7-acf5217f6c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:39.475888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e8a4c-8af3-4631-812e-56027cbb6745 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:39.477358 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e8a4c-8af3-4631-812e-56027cbb6745 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8de87d8f-55a3-4946-86a7-acf5217f6c6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:39.482303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:39.484232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:39.485862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 213/422] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:39.489290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de87d8f-55a3-4946-86a7-acf5217f6c6b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:39.511771 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:39.588693 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 210/423] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 18:03:39.601019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e8a4c-8af3-4631-812e-56027cbb6745 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:39.623525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:39.708068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de87d8f-55a3-4946-86a7-acf5217f6c6b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:39.761758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0242c-500e-4e41-88f7-f05bc341f6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:39.762643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0242c-500e-4e41-88f7-f05bc341f6cf None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:39.763121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0242c-500e-4e41-88f7-f05bc341f6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:39.763622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0242c-500e-4e41-88f7-f05bc341f6cf None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:39.764556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0242c-500e-4e41-88f7-f05bc341f6cf None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:39.764968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0242c-500e-4e41-88f7-f05bc341f6cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:39.766616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0242c-500e-4e41-88f7-f05bc341f6cf None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:39.776668 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 214/424] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:39.782952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0242c-500e-4e41-88f7-f05bc341f6cf None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:39.784811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e8a4c-8af3-4631-812e-56027cbb6745 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:39.829226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:39.844513 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf70e0-84e7-42b8-acda-c1798ca72f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:39.845472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf70e0-84e7-42b8-acda-c1798ca72f5e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:39.845769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf70e0-84e7-42b8-acda-c1798ca72f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:39.846043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf70e0-84e7-42b8-acda-c1798ca72f5e None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:39.847225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf70e0-84e7-42b8-acda-c1798ca72f5e None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:39.848874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf70e0-84e7-42b8-acda-c1798ca72f5e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:39.850894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 211/425] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:39 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:39.856888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf70e0-84e7-42b8-acda-c1798ca72f5e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:39.917456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0242c-500e-4e41-88f7-f05bc341f6cf None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:39.946328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:39.955896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b9d81-89bd-4eda-a890-e799b7affe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:39.956541 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b9d81-89bd-4eda-a890-e799b7affe55 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:39.956860 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b9d81-89bd-4eda-a890-e799b7affe55 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:39.957134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b9d81-89bd-4eda-a890-e799b7affe55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:39.979200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf70e0-84e7-42b8-acda-c1798ca72f5e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:40.048677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b9d81-89bd-4eda-a890-e799b7affe55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:40.050184 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b9d81-89bd-4eda-a890-e799b7affe55 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:40.060703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b733e2-24f3-475e-9d5e-e14f3cc265a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:40.061578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b733e2-24f3-475e-9d5e-e14f3cc265a6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:40.061893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b733e2-24f3-475e-9d5e-e14f3cc265a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:40.062262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b733e2-24f3-475e-9d5e-e14f3cc265a6 None admin] PATH_INFO: `/v3/groups/086a00dba351454e91ecb6633bf9eaa5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:40.063557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b733e2-24f3-475e-9d5e-e14f3cc265a6 None admin] RBAC: Authorizing `identity:get_group(group_id=086a00dba351454e91ecb6633bf9eaa5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:40.065006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b733e2-24f3-475e-9d5e-e14f3cc265a6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:40.079463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 212/426] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:39 2022] GET /identity/v3/groups/086a00dba351454e91ecb6633bf9eaa5 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:40.085635 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b733e2-24f3-475e-9d5e-e14f3cc265a6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:40.254331 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:40.352038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b733e2-24f3-475e-9d5e-e14f3cc265a6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:40.364512 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 215/427] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:40.377015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b9d81-89bd-4eda-a890-e799b7affe55 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:40.483637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ea236-e0e0-408d-bdb9-d59afca11919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:40.484300 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ea236-e0e0-408d-bdb9-d59afca11919 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:40.484572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ea236-e0e0-408d-bdb9-d59afca11919 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:40.484841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ea236-e0e0-408d-bdb9-d59afca11919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:40.485667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ea236-e0e0-408d-bdb9-d59afca11919 None admin] RBAC: Adding query filter params (group.id=086a00dba351454e91ecb6633bf9eaa5, role.id=506b3cf33324478abe0f72a9d408aa54, scope.project.id=f5b2e96e553d496d96e8f916fb522477) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:40.486225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ea236-e0e0-408d-bdb9-d59afca11919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:40.488252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ea236-e0e0-408d-bdb9-d59afca11919 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:40.500824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:40.516943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 213/428] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 18:03:40 2022] GET /identity/v3/role_assignments?group.id=086a00dba351454e91ecb6633bf9eaa5&scope.project.id=f5b2e96e553d496d96e8f916fb522477&role.id=506b3cf33324478abe0f72a9d408aa54 => generated 611 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:40.676944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b9d81-89bd-4eda-a890-e799b7affe55 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:40.739734 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1bcef-5e4e-4870-bdd0-85ede01e67af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:40.741041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1bcef-5e4e-4870-bdd0-85ede01e67af None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:40.741459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1bcef-5e4e-4870-bdd0-85ede01e67af None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:40.741901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1bcef-5e4e-4870-bdd0-85ede01e67af None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:40.743712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1bcef-5e4e-4870-bdd0-85ede01e67af None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:40.746134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1bcef-5e4e-4870-bdd0-85ede01e67af None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:40.750590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 216/429] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:40 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:40.756033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ea236-e0e0-408d-bdb9-d59afca11919 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:40.778046 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbfbcb-fb52-4676-a8ca-0fcb79097aff None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:40.778419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbfbcb-fb52-4676-a8ca-0fcb79097aff None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:40.778753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbfbcb-fb52-4676-a8ca-0fcb79097aff None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:40.780785 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 217/430] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:40.787458 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d00c18-39fa-4582-aa57-6b0e9cd61ef1 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:40.787794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d00c18-39fa-4582-aa57-6b0e9cd61ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:40.788121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d00c18-39fa-4582-aa57-6b0e9cd61ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:40.821074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e5d00c18-39fa-4582-aa57-6b0e9cd61ef1 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:40.822337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e5d00c18-39fa-4582-aa57-6b0e9cd61ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:40.936363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 218/431] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:40.937210 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ea236-e0e0-408d-bdb9-d59afca11919 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:40.966712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d00c18-39fa-4582-aa57-6b0e9cd61ef1 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:40.973104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:40.985079 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b382db-5317-44de-91d8-a1daa3e821ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:40.986009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b382db-5317-44de-91d8-a1daa3e821ee None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:40.986297 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b382db-5317-44de-91d8-a1daa3e821ee None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:40.986570 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b382db-5317-44de-91d8-a1daa3e821ee None admin] PATH_INFO: `/v3/groups/086a00dba351454e91ecb6633bf9eaa5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:40.987864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b382db-5317-44de-91d8-a1daa3e821ee None admin] RBAC: Authorizing `identity:get_group(group_id=086a00dba351454e91ecb6633bf9eaa5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:40.989328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b382db-5317-44de-91d8-a1daa3e821ee None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:40.993046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 214/432] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:40 2022] GET /identity/v3/groups/086a00dba351454e91ecb6633bf9eaa5 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:40.997394 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:40.999337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b382db-5317-44de-91d8-a1daa3e821ee None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:41.086894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d00c18-39fa-4582-aa57-6b0e9cd61ef1 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:41.118043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b382db-5317-44de-91d8-a1daa3e821ee None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:41.122609 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de12bce-592e-44bd-867b-0a01cda703f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:41.123522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de12bce-592e-44bd-867b-0a01cda703f7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:41.123800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de12bce-592e-44bd-867b-0a01cda703f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:41.124094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de12bce-592e-44bd-867b-0a01cda703f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:41.134185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de12bce-592e-44bd-867b-0a01cda703f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8de12bce-592e-44bd-867b-0a01cda703f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:41.143100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:41.144081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:41.144854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 219/433] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:03:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:41.146986 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05daaf31-f36e-478c-8e91-e72a2d6f991e None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:41.147268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05daaf31-f36e-478c-8e91-e72a2d6f991e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:41.147599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05daaf31-f36e-478c-8e91-e72a2d6f991e None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:41.149224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 220/434] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:41.150134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfbf86-8fd0-4e47-8c45-c25119894519 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:41.150481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfbf86-8fd0-4e47-8c45-c25119894519 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:41.150675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfbf86-8fd0-4e47-8c45-c25119894519 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:41.152501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 221/435] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:41.154964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfbf86-8fd0-4e47-8c45-c25119894519 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:41.156710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c81c5-8fd4-45b8-89cb-3a15d11c95ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:41.157344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c81c5-8fd4-45b8-89cb-3a15d11c95ee None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:41.157618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c81c5-8fd4-45b8-89cb-3a15d11c95ee None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:41.157907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c81c5-8fd4-45b8-89cb-3a15d11c95ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:41.158760 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c81c5-8fd4-45b8-89cb-3a15d11c95ee None admin] RBAC: Adding query filter params (group.id=086a00dba351454e91ecb6633bf9eaa5, role.id=e1a5192e7abc4046bc2c0673176ea1b6, scope.project.id=fea9cdff50914413b637910a718a6cd5) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:41.159034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c81c5-8fd4-45b8-89cb-3a15d11c95ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:41.160732 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c81c5-8fd4-45b8-89cb-3a15d11c95ee None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:41.167496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 215/436] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 18:03:40 2022] GET /identity/v3/role_assignments?group.id=086a00dba351454e91ecb6633bf9eaa5&scope.project.id=fea9cdff50914413b637910a718a6cd5&role.id=e1a5192e7abc4046bc2c0673176ea1b6 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:41.169281 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238c592-5dc0-4d4f-986d-11a13207e5b5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:41.169565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238c592-5dc0-4d4f-986d-11a13207e5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:41.169865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238c592-5dc0-4d4f-986d-11a13207e5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:41.189797 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-3238c592-5dc0-4d4f-986d-11a13207e5b5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:41.191836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3238c592-5dc0-4d4f-986d-11a13207e5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:41.260057 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:41.276976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:41.279924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 216/437] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:41.284410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b781a08-16c7-46b0-b577-619a80551553 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:41.285852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b781a08-16c7-46b0-b577-619a80551553 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:41.286360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b781a08-16c7-46b0-b577-619a80551553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:41.301326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfbf86-8fd0-4e47-8c45-c25119894519 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:41.312259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-3b781a08-16c7-46b0-b577-619a80551553 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:41.317263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3b781a08-16c7-46b0-b577-619a80551553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:41.352929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e29ee-e53f-4732-aa12-417d7429fc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:41.353508 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e29ee-e53f-4732-aa12-417d7429fc55 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:41.353813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e29ee-e53f-4732-aa12-417d7429fc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:41.354261 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e29ee-e53f-4732-aa12-417d7429fc55 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:41.354842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e29ee-e53f-4732-aa12-417d7429fc55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:41.355140 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e29ee-e53f-4732-aa12-417d7429fc55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:41.357391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e29ee-e53f-4732-aa12-417d7429fc55 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:41.367532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 222/438] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:41.368650 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e29ee-e53f-4732-aa12-417d7429fc55 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:41.405367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 217/439] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:41.406180 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b781a08-16c7-46b0-b577-619a80551553 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:41.494231 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e29ee-e53f-4732-aa12-417d7429fc55 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:41.510318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b781a08-16c7-46b0-b577-619a80551553 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:41.530215 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d954b58-44e7-4986-a784-f6c579c6e397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:41.531010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d954b58-44e7-4986-a784-f6c579c6e397 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:41.531290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d954b58-44e7-4986-a784-f6c579c6e397 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:41.531559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d954b58-44e7-4986-a784-f6c579c6e397 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:41.532357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:41.542600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d954b58-44e7-4986-a784-f6c579c6e397 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:41.546853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d4721-806d-45ec-b38c-e5b2e8b45fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:41.547494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d4721-806d-45ec-b38c-e5b2e8b45fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:41.547791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d4721-806d-45ec-b38c-e5b2e8b45fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:41.548046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d4721-806d-45ec-b38c-e5b2e8b45fd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8d954b58-44e7-4986-a784-f6c579c6e397 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:41.550300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:41.551152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:41.551954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 223/440] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:03:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:41.552853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d954b58-44e7-4986-a784-f6c579c6e397 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:41.562674 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:41.622264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d4721-806d-45ec-b38c-e5b2e8b45fd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:41.623765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d4721-806d-45ec-b38c-e5b2e8b45fd7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:41.659322 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d954b58-44e7-4986-a784-f6c579c6e397 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:41.696000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde817b6-ab18-43b2-8d5a-0cf25fb2a819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:41.696784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde817b6-ab18-43b2-8d5a-0cf25fb2a819 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:41.697063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde817b6-ab18-43b2-8d5a-0cf25fb2a819 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:41.697331 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde817b6-ab18-43b2-8d5a-0cf25fb2a819 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:41.707700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde817b6-ab18-43b2-8d5a-0cf25fb2a819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-fde817b6-ab18-43b2-8d5a-0cf25fb2a819 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:41.715338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:41.716378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:41.717170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 224/441] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:41.718181 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde817b6-ab18-43b2-8d5a-0cf25fb2a819 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:41.742336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 218/442] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:41.742336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d4721-806d-45ec-b38c-e5b2e8b45fd7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:41.776205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:41.797039 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:41.837691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde817b6-ab18-43b2-8d5a-0cf25fb2a819 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:41.864283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d4721-806d-45ec-b38c-e5b2e8b45fd7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:41.873417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeec4d4-db66-44c1-a70a-1b89af64ff54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:41.874061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeec4d4-db66-44c1-a70a-1b89af64ff54 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:41.874349 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeec4d4-db66-44c1-a70a-1b89af64ff54 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:41.874624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeec4d4-db66-44c1-a70a-1b89af64ff54 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:41.875358 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeec4d4-db66-44c1-a70a-1b89af64ff54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:41.875661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeec4d4-db66-44c1-a70a-1b89af64ff54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:41.877060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeec4d4-db66-44c1-a70a-1b89af64ff54 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:41.885372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 225/443] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:41.886197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeec4d4-db66-44c1-a70a-1b89af64ff54 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:41.904151 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80f375-1f27-4985-96d1-b96535a541dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:41.904815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80f375-1f27-4985-96d1-b96535a541dd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:41.905091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80f375-1f27-4985-96d1-b96535a541dd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:41.905360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80f375-1f27-4985-96d1-b96535a541dd None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:41.906127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80f375-1f27-4985-96d1-b96535a541dd None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:41.906435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80f375-1f27-4985-96d1-b96535a541dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:41.907789 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80f375-1f27-4985-96d1-b96535a541dd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:41.916568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 219/444] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:03:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:41.917416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80f375-1f27-4985-96d1-b96535a541dd None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:41.998574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeec4d4-db66-44c1-a70a-1b89af64ff54 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:42.007356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:42.025997 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:42.026397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80f375-1f27-4985-96d1-b96535a541dd None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:42.040364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db3e65-fa87-4e51-b67e-1e4f4f069638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:42.041223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db3e65-fa87-4e51-b67e-1e4f4f069638 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:42.041501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db3e65-fa87-4e51-b67e-1e4f4f069638 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:42.041801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db3e65-fa87-4e51-b67e-1e4f4f069638 None admin] PATH_INFO: `/v3/projects/fea9cdff50914413b637910a718a6cd5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:42.042996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db3e65-fa87-4e51-b67e-1e4f4f069638 None admin] RBAC: Authorizing `identity:get_project(project_id=fea9cdff50914413b637910a718a6cd5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:42.044462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db3e65-fa87-4e51-b67e-1e4f4f069638 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:42.046326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 226/445] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:41 2022] GET /identity/v3/projects/fea9cdff50914413b637910a718a6cd5 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:42.047076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db3e65-fa87-4e51-b67e-1e4f4f069638 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:42.067776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7270ac-4431-4674-9904-b3c8305cf1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:42.068418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7270ac-4431-4674-9904-b3c8305cf1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:42.068687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7270ac-4431-4674-9904-b3c8305cf1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:42.068955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7270ac-4431-4674-9904-b3c8305cf1e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:42.148237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7270ac-4431-4674-9904-b3c8305cf1e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:42.150132 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7270ac-4431-4674-9904-b3c8305cf1e8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:42.157109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db3e65-fa87-4e51-b67e-1e4f4f069638 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:42.197604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452618a4-b8e0-4071-b301-3b862f83d78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:42.198273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452618a4-b8e0-4071-b301-3b862f83d78a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:42.198643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452618a4-b8e0-4071-b301-3b862f83d78a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:42.199011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452618a4-b8e0-4071-b301-3b862f83d78a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:42.237540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:42.258779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 220/446] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:42.260068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7270ac-4431-4674-9904-b3c8305cf1e8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:42.279499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:42.288160 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452618a4-b8e0-4071-b301-3b862f83d78a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:42.289639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452618a4-b8e0-4071-b301-3b862f83d78a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:42.376107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7270ac-4431-4674-9904-b3c8305cf1e8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:42.397251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 227/447] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:42.398140 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452618a4-b8e0-4071-b301-3b862f83d78a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:42.413430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8ed0b-f2a4-4bfc-970a-2d9fd9638606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:42.414206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8ed0b-f2a4-4bfc-970a-2d9fd9638606 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:42.414472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8ed0b-f2a4-4bfc-970a-2d9fd9638606 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:42.414739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8ed0b-f2a4-4bfc-970a-2d9fd9638606 None admin] PATH_INFO: `/v3/groups/086a00dba351454e91ecb6633bf9eaa5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:42.415913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8ed0b-f2a4-4bfc-970a-2d9fd9638606 None admin] RBAC: Authorizing `identity:get_group(group_id=086a00dba351454e91ecb6633bf9eaa5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:42.417321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8ed0b-f2a4-4bfc-970a-2d9fd9638606 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:42.425948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 221/448] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:42 2022] GET /identity/v3/groups/086a00dba351454e91ecb6633bf9eaa5 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:42.425948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8ed0b-f2a4-4bfc-970a-2d9fd9638606 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:42.478643 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:42.501719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:42.507568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452618a4-b8e0-4071-b301-3b862f83d78a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:42.532587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8ed0b-f2a4-4bfc-970a-2d9fd9638606 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:42.545858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74daa6-55e2-4fe3-ae4e-e8b6f07a2866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:42.546707 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74daa6-55e2-4fe3-ae4e-e8b6f07a2866 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:42.546977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74daa6-55e2-4fe3-ae4e-e8b6f07a2866 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:42.547242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74daa6-55e2-4fe3-ae4e-e8b6f07a2866 None admin] PATH_INFO: `/v3/projects/bbbdb1cacacf4ba5a41a00d6d478efbe` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:42.548399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74daa6-55e2-4fe3-ae4e-e8b6f07a2866 None admin] RBAC: Authorizing `identity:get_project(project_id=bbbdb1cacacf4ba5a41a00d6d478efbe)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:42.550055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74daa6-55e2-4fe3-ae4e-e8b6f07a2866 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:42.551583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 228/449] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:42 2022] GET /identity/v3/projects/bbbdb1cacacf4ba5a41a00d6d478efbe => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:42.552344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74daa6-55e2-4fe3-ae4e-e8b6f07a2866 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:42.567816 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915aea0-0ac5-4e82-86cf-31e08491d30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:42.568669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915aea0-0ac5-4e82-86cf-31e08491d30d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:42.568948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915aea0-0ac5-4e82-86cf-31e08491d30d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:42.569218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915aea0-0ac5-4e82-86cf-31e08491d30d None admin] PATH_INFO: `/v3/projects/f5b2e96e553d496d96e8f916fb522477` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:42.570409 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915aea0-0ac5-4e82-86cf-31e08491d30d None admin] RBAC: Authorizing `identity:get_project(project_id=f5b2e96e553d496d96e8f916fb522477)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:42.571829 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915aea0-0ac5-4e82-86cf-31e08491d30d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:42.573507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 222/450] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:03:42 2022] GET /identity/v3/projects/f5b2e96e553d496d96e8f916fb522477 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:42.574286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915aea0-0ac5-4e82-86cf-31e08491d30d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:42.658095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74daa6-55e2-4fe3-ae4e-e8b6f07a2866 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:42.680824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915aea0-0ac5-4e82-86cf-31e08491d30d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:42.692416 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc94c3e-fd6c-48d7-8bf2-7944d7886e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:42.693069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc94c3e-fd6c-48d7-8bf2-7944d7886e3f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:42.693349 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc94c3e-fd6c-48d7-8bf2-7944d7886e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:42.693644 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc94c3e-fd6c-48d7-8bf2-7944d7886e3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:42.694474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc94c3e-fd6c-48d7-8bf2-7944d7886e3f None admin] RBAC: Adding query filter params (group.id=086a00dba351454e91ecb6633bf9eaa5, role.id=506b3cf33324478abe0f72a9d408aa54, scope.project.id=fea9cdff50914413b637910a718a6cd5) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:42.694784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc94c3e-fd6c-48d7-8bf2-7944d7886e3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:42.696240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc94c3e-fd6c-48d7-8bf2-7944d7886e3f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:42.700607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:42.713757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 229/451] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 18:03:42 2022] GET /identity/v3/role_assignments?group.id=086a00dba351454e91ecb6633bf9eaa5&scope.project.id=fea9cdff50914413b637910a718a6cd5&role.id=506b3cf33324478abe0f72a9d408aa54 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:42.713757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc94c3e-fd6c-48d7-8bf2-7944d7886e3f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:42.724605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca114b-395a-421d-bb40-e3d20a00b03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:42.725799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca114b-395a-421d-bb40-e3d20a00b03c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:42.726426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca114b-395a-421d-bb40-e3d20a00b03c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:42.726865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca114b-395a-421d-bb40-e3d20a00b03c None admin] PATH_INFO: `/v3/groups/ee8d6a358ca94bb798b47effa5c74506` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:42.728877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca114b-395a-421d-bb40-e3d20a00b03c None admin] RBAC: Authorizing `identity:get_group(group_id=ee8d6a358ca94bb798b47effa5c74506)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:42.730672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca114b-395a-421d-bb40-e3d20a00b03c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:42.733928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 223/452] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:42 2022] GET /identity/v3/groups/ee8d6a358ca94bb798b47effa5c74506 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:42.737812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:42.742774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca114b-395a-421d-bb40-e3d20a00b03c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:42.836585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc94c3e-fd6c-48d7-8bf2-7944d7886e3f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:42.880090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51155863-2bed-40c4-9a9d-1f6c13ca78e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:42.880965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51155863-2bed-40c4-9a9d-1f6c13ca78e2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:42.881281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51155863-2bed-40c4-9a9d-1f6c13ca78e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:42.881521 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51155863-2bed-40c4-9a9d-1f6c13ca78e2 None admin] PATH_INFO: `/v3/groups/086a00dba351454e91ecb6633bf9eaa5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:42.882826 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51155863-2bed-40c4-9a9d-1f6c13ca78e2 None admin] RBAC: Authorizing `identity:get_group(group_id=086a00dba351454e91ecb6633bf9eaa5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:42.884317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51155863-2bed-40c4-9a9d-1f6c13ca78e2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:42.892298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 230/453] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 18:03:42 2022] GET /identity/v3/groups/086a00dba351454e91ecb6633bf9eaa5 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:42.895186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51155863-2bed-40c4-9a9d-1f6c13ca78e2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:42.899086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca114b-395a-421d-bb40-e3d20a00b03c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:42.947053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93234db2-6c82-43dc-b282-b4f2720ab7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:42.947628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93234db2-6c82-43dc-b282-b4f2720ab7fd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:42.947903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93234db2-6c82-43dc-b282-b4f2720ab7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:42.948172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93234db2-6c82-43dc-b282-b4f2720ab7fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:42.948990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93234db2-6c82-43dc-b282-b4f2720ab7fd None admin] RBAC: Adding query filter params (group.id=ee8d6a358ca94bb798b47effa5c74506, role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=bbbdb1cacacf4ba5a41a00d6d478efbe) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:42.949297 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93234db2-6c82-43dc-b282-b4f2720ab7fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:42.950964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93234db2-6c82-43dc-b282-b4f2720ab7fd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:42.957367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 224/454] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 18:03:42 2022] GET /identity/v3/role_assignments?group.id=ee8d6a358ca94bb798b47effa5c74506&scope.project.id=bbbdb1cacacf4ba5a41a00d6d478efbe&role.id=584beecb022b422f820e77f7623ada5e => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:43.030880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:43.037909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51155863-2bed-40c4-9a9d-1f6c13ca78e2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:43.096347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802543b9-ede5-4a9d-8adc-b998f3877c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:43.097486 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802543b9-ede5-4a9d-8adc-b998f3877c1b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:43.098009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802543b9-ede5-4a9d-8adc-b998f3877c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:43.098478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802543b9-ede5-4a9d-8adc-b998f3877c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:43.100056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802543b9-ede5-4a9d-8adc-b998f3877c1b None admin] RBAC: Adding query filter params (group.id=086a00dba351454e91ecb6633bf9eaa5, role.id=e1a5192e7abc4046bc2c0673176ea1b6, scope.project.id=f5b2e96e553d496d96e8f916fb522477) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:43.100600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802543b9-ede5-4a9d-8adc-b998f3877c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:43.103306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802543b9-ede5-4a9d-8adc-b998f3877c1b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:43.125854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 231/455] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 18:03:42 2022] GET /identity/v3/role_assignments?group.id=086a00dba351454e91ecb6633bf9eaa5&scope.project.id=f5b2e96e553d496d96e8f916fb522477&role.id=e1a5192e7abc4046bc2c0673176ea1b6 => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:46.703600 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa222489-f147-445b-afe7-f8c0da6162af None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:46.704653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa222489-f147-445b-afe7-f8c0da6162af None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:46.705142 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa222489-f147-445b-afe7-f8c0da6162af None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:46.707000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 225/456] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:46.731847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a4320-f515-42ed-8783-30b771287d9d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:46.732820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a4320-f515-42ed-8783-30b771287d9d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:46.733307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a4320-f515-42ed-8783-30b771287d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:46.752467 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-0b6a4320-f515-42ed-8783-30b771287d9d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:46.754022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0b6a4320-f515-42ed-8783-30b771287d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:46.848118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 232/457] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:46.908449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa222489-f147-445b-afe7-f8c0da6162af None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:46.986134 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:47.046899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa222489-f147-445b-afe7-f8c0da6162af None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:47.084264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30c54c-fb8a-4c79-bcc9-8ba617ab0b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:47.085444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30c54c-fb8a-4c79-bcc9-8ba617ab0b77 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:47.085980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30c54c-fb8a-4c79-bcc9-8ba617ab0b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:47.086465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30c54c-fb8a-4c79-bcc9-8ba617ab0b77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:47.096803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e223399-f63b-4f5e-b94c-127819fd1c81 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:47.097409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e223399-f63b-4f5e-b94c-127819fd1c81 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:47.097843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30c54c-fb8a-4c79-bcc9-8ba617ab0b77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:47.098184 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e223399-f63b-4f5e-b94c-127819fd1c81 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:47.099504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30c54c-fb8a-4c79-bcc9-8ba617ab0b77 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:47.100344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 233/458] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:47.105396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e577d-9b06-488a-af49-6b49e1d065eb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:47.106558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e577d-9b06-488a-af49-6b49e1d065eb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:47.107049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e577d-9b06-488a-af49-6b49e1d065eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-5b30c54c-fb8a-4c79-bcc9-8ba617ab0b77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:47.118662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:47.119583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:47.120465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:47.120465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:47.120465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:47.120465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:47.120465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 226/459] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:47.128755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30c54c-fb8a-4c79-bcc9-8ba617ab0b77 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:47.154791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-eb5e577d-9b06-488a-af49-6b49e1d065eb None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:47.156363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-eb5e577d-9b06-488a-af49-6b49e1d065eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:47.246339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30c54c-fb8a-4c79-bcc9-8ba617ab0b77 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:47.259066 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:47.303033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906019c2-759d-42c8-8083-2abf5fa432b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:47.304091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906019c2-759d-42c8-8083-2abf5fa432b8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:47.305025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906019c2-759d-42c8-8083-2abf5fa432b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:47.305516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906019c2-759d-42c8-8083-2abf5fa432b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:47.306775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906019c2-759d-42c8-8083-2abf5fa432b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:47.307521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906019c2-759d-42c8-8083-2abf5fa432b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:47.309803 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906019c2-759d-42c8-8083-2abf5fa432b8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:47.324946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 227/460] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:47.326645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2ca5c-7c43-4d70-90e3-b65ed3fca03d None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:47.327317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2ca5c-7c43-4d70-90e3-b65ed3fca03d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:47.327725 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2ca5c-7c43-4d70-90e3-b65ed3fca03d None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:47.329378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 228/461] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:47.333509 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 234/462] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:47.334488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2ca5c-7c43-4d70-90e3-b65ed3fca03d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:47.336733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630b834-3da0-4e28-9d48-b8985f6ab77f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:47.337219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630b834-3da0-4e28-9d48-b8985f6ab77f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:47.337681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630b834-3da0-4e28-9d48-b8985f6ab77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:47.369101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-0630b834-3da0-4e28-9d48-b8985f6ab77f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:47.370542 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0630b834-3da0-4e28-9d48-b8985f6ab77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:47.395597 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:47.471543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 235/463] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:47.472682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630b834-3da0-4e28-9d48-b8985f6ab77f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:47.487604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2ca5c-7c43-4d70-90e3-b65ed3fca03d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:47.534340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36252e-aea9-4693-96c7-ce97caa711aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:47.535704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36252e-aea9-4693-96c7-ce97caa711aa None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:47.536210 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36252e-aea9-4693-96c7-ce97caa711aa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:47.536691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36252e-aea9-4693-96c7-ce97caa711aa None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:47.537600 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36252e-aea9-4693-96c7-ce97caa711aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:47.539451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36252e-aea9-4693-96c7-ce97caa711aa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:47.568277 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 229/464] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:03:47 2022] POST /identity/v3/users => generated 248 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:47.569908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36252e-aea9-4693-96c7-ce97caa711aa None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:47.589975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630b834-3da0-4e28-9d48-b8985f6ab77f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:47.604393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:47.656484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004b94e-0686-445a-8b0f-3325770cea0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:47.656963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004b94e-0686-445a-8b0f-3325770cea0a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:47.657249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004b94e-0686-445a-8b0f-3325770cea0a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:47.657519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004b94e-0686-445a-8b0f-3325770cea0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:47.670950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004b94e-0686-445a-8b0f-3325770cea0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:47.672534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004b94e-0686-445a-8b0f-3325770cea0a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-5004b94e-0686-445a-8b0f-3325770cea0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:47.693651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:47.694606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:47.695450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:47.695450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:47.695450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:47.695450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:47.695450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 236/465] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:47.696651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc3a00-cd36-4364-b152-e0978f02b655 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:47.696955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc3a00-cd36-4364-b152-e0978f02b655 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:47.697226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc3a00-cd36-4364-b152-e0978f02b655 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:47.702898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 237/466] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:47.704043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc3a00-cd36-4364-b152-e0978f02b655 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:47.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:47.717970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36252e-aea9-4693-96c7-ce97caa711aa None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:47.757481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936ae04-3428-4869-9f45-65cc457fbd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:47.758705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936ae04-3428-4869-9f45-65cc457fbd31 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:47.759194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936ae04-3428-4869-9f45-65cc457fbd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:47.759682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936ae04-3428-4869-9f45-65cc457fbd31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:47.772446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2936ae04-3428-4869-9f45-65cc457fbd31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:47.774328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2936ae04-3428-4869-9f45-65cc457fbd31 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-2936ae04-3428-4869-9f45-65cc457fbd31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:47.795764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:47.796631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:47.797464 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:47.797464 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:47.797464 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:47.797464 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:47.798448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 230/467] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:47.801825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c87a9e-6976-4b35-9ef6-d8da800f2787 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:47.801825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c87a9e-6976-4b35-9ef6-d8da800f2787 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:47.801825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c87a9e-6976-4b35-9ef6-d8da800f2787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:47.818960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-39c87a9e-6976-4b35-9ef6-d8da800f2787 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:47.820146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-39c87a9e-6976-4b35-9ef6-d8da800f2787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:47.830317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc3a00-cd36-4364-b152-e0978f02b655 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:47.875214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd9b19-5dd0-49f0-9bae-415f9e4f8ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:47.876279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd9b19-5dd0-49f0-9bae-415f9e4f8ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:47.876565 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd9b19-5dd0-49f0-9bae-415f9e4f8ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:47.876836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd9b19-5dd0-49f0-9bae-415f9e4f8ac7 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:47.878085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd9b19-5dd0-49f0-9bae-415f9e4f8ac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:47.878536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd9b19-5dd0-49f0-9bae-415f9e4f8ac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:47.880130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd9b19-5dd0-49f0-9bae-415f9e4f8ac7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:47.896977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 238/468] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:47.897878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd9b19-5dd0-49f0-9bae-415f9e4f8ac7 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:47.900405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:47.907844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 231/469] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:47.908813 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c87a9e-6976-4b35-9ef6-d8da800f2787 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:47.977260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:48.025647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd9b19-5dd0-49f0-9bae-415f9e4f8ac7 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:48.031408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c87a9e-6976-4b35-9ef6-d8da800f2787 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:48.072390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334803f-10b2-40e4-a817-d523613cf59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:48.072575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8a559-fc23-40fd-9369-83097fde29b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:48.072969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334803f-10b2-40e4-a817-d523613cf59c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:48.073073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8a559-fc23-40fd-9369-83097fde29b6 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:48.073236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334803f-10b2-40e4-a817-d523613cf59c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:48.073345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8a559-fc23-40fd-9369-83097fde29b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:48.073514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334803f-10b2-40e4-a817-d523613cf59c None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:48.073608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8a559-fc23-40fd-9369-83097fde29b6 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:48.074271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334803f-10b2-40e4-a817-d523613cf59c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:48.074370 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8a559-fc23-40fd-9369-83097fde29b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:48.074575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334803f-10b2-40e4-a817-d523613cf59c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:48.076367 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334803f-10b2-40e4-a817-d523613cf59c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:48.076492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8a559-fc23-40fd-9369-83097fde29b6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:48.092587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 239/470] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:48.093574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334803f-10b2-40e4-a817-d523613cf59c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:48.108945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 232/471] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:03:47 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:48.110576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8a559-fc23-40fd-9369-83097fde29b6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:48.180838 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:48.290945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8a559-fc23-40fd-9369-83097fde29b6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:48.295555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334803f-10b2-40e4-a817-d523613cf59c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:48.334728 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:48.344251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bececbff-3cfe-4479-b1b0-863cdeca0292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:48.344890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bececbff-3cfe-4479-b1b0-863cdeca0292 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:48.345169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bececbff-3cfe-4479-b1b0-863cdeca0292 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:48.345458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bececbff-3cfe-4479-b1b0-863cdeca0292 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:48.346919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bececbff-3cfe-4479-b1b0-863cdeca0292 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:48.348374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bececbff-3cfe-4479-b1b0-863cdeca0292 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:48.351144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0eee4-69c9-46cf-ab3c-20ef9ff2ecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:48.352129 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0eee4-69c9-46cf-ab3c-20ef9ff2ecc6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:48.352404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0eee4-69c9-46cf-ab3c-20ef9ff2ecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:48.352668 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0eee4-69c9-46cf-ab3c-20ef9ff2ecc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:48.366763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0eee4-69c9-46cf-ab3c-20ef9ff2ecc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:48.368254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0eee4-69c9-46cf-ab3c-20ef9ff2ecc6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:48.375261 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 233/472] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:03:48 2022] POST /identity/v3/users => generated 246 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:48.376850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99534be-e505-4ce9-83fc-0cffe7e87d3e None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:48.377177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99534be-e505-4ce9-83fc-0cffe7e87d3e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:48.377561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99534be-e505-4ce9-83fc-0cffe7e87d3e None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:48.379456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 234/473] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:48.385458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa60d29-d08f-4612-9dbf-da1d7f09847d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:48.385846 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa60d29-d08f-4612-9dbf-da1d7f09847d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:48.386158 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa60d29-d08f-4612-9dbf-da1d7f09847d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-fac0eee4-69c9-46cf-ab3c-20ef9ff2ecc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:48.389308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:48.390288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:48.391139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:48.391139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:48.391139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:48.391139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:48.391139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 240/474] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:48.398166 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0eee4-69c9-46cf-ab3c-20ef9ff2ecc6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:48.414221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1aa60d29-d08f-4612-9dbf-da1d7f09847d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:48.415589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1aa60d29-d08f-4612-9dbf-da1d7f09847d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:48.505899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 235/475] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:48.520365 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0eee4-69c9-46cf-ab3c-20ef9ff2ecc6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:48.533768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa60d29-d08f-4612-9dbf-da1d7f09847d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:48.565822 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25f6c6-bb3f-4fe3-92fe-08768a300039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:48.566494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25f6c6-bb3f-4fe3-92fe-08768a300039 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:48.566770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25f6c6-bb3f-4fe3-92fe-08768a300039 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:48.567034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25f6c6-bb3f-4fe3-92fe-08768a300039 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:48.567760 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a25f6c6-bb3f-4fe3-92fe-08768a300039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:48.568060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a25f6c6-bb3f-4fe3-92fe-08768a300039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:48.569472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a25f6c6-bb3f-4fe3-92fe-08768a300039 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:48.584373 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 241/476] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:48.591939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25f6c6-bb3f-4fe3-92fe-08768a300039 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:48.594795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:48.637981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:48.683787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa60d29-d08f-4612-9dbf-da1d7f09847d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:48.728222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed694c4-f56e-478d-ba55-b3c2ddbc92a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:48.730171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed694c4-f56e-478d-ba55-b3c2ddbc92a5 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:48.730171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed694c4-f56e-478d-ba55-b3c2ddbc92a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:48.730171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed694c4-f56e-478d-ba55-b3c2ddbc92a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:48.730416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed694c4-f56e-478d-ba55-b3c2ddbc92a5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:48.732074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25f6c6-bb3f-4fe3-92fe-08768a300039 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:48.732539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed694c4-f56e-478d-ba55-b3c2ddbc92a5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:48.750009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.sql.core [None req-6ed694c4-f56e-478d-ba55-b3c2ddbc92a5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 18:03:48.750009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 18:03:48.750009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [parameters: {'id': 'e91202db451b41e79e3d3e68f3a06d6d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 18:03:48.750009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80324) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 18:03:48.779053 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d706a6-1a3c-48cd-85e6-dc1b1689f9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:48.780108 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d706a6-1a3c-48cd-85e6-dc1b1689f9fb None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:48.780692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d706a6-1a3c-48cd-85e6-dc1b1689f9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:48.781164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d706a6-1a3c-48cd-85e6-dc1b1689f9fb None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:48.782268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d706a6-1a3c-48cd-85e6-dc1b1689f9fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:48.784185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d706a6-1a3c-48cd-85e6-dc1b1689f9fb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-6ed694c4-f56e-478d-ba55-b3c2ddbc92a5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application result = self._query(query) May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application conn.query(q) May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application result.read() May 26 18:03:48.784996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return ref.to_dict() May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 18:03:48.786196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application next(self.gen) May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application yield resource May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application next(self.gen) May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self.session.rollback() May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self.force_reraise() May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise self.value May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application session.commit() May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 18:03:48.787178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._prepare_impl() May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self.session.flush() May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._flush(objects) May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application compat.raise_( May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application flush_context.execute() May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rec.execute(self) May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 18:03:48.788085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 18:03:48.789500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application result = self._query(query) May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application conn.query(q) May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application result.read() May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 18:03:48.791520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [parameters: {'id': 'e91202db451b41e79e3d3e68f3a06d6d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:48.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 18:03:48.795794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:48.798786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 236/477] 10.222.0.46 () {60 vars in 1131 bytes} [Thu May 26 18:03:48 2022] POST /identity/v3/roles => generated 144 bytes in 254 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:48.798786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed694c4-f56e-478d-ba55-b3c2ddbc92a5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:48.838593 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 242/478] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:03:48 2022] POST /identity/v3/users => generated 249 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:03:48.995089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed694c4-f56e-478d-ba55-b3c2ddbc92a5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:49.014441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:49.086379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c23ad0-9235-438f-977c-c7822d773a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:49.087592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c23ad0-9235-438f-977c-c7822d773a63 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:49.088008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c23ad0-9235-438f-977c-c7822d773a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:49.088416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c23ad0-9235-438f-977c-c7822d773a63 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:49.105217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c23ad0-9235-438f-977c-c7822d773a63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-76c23ad0-9235-438f-977c-c7822d773a63 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:49.116634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:49.117590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:49.118435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 237/479] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:03:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:49.124849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d706a6-1a3c-48cd-85e6-dc1b1689f9fb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:49.260775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:49.283301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d706a6-1a3c-48cd-85e6-dc1b1689f9fb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:49.347397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215061db-a428-49e7-8943-a907537538a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:49.348389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215061db-a428-49e7-8943-a907537538a5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:49.348816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215061db-a428-49e7-8943-a907537538a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:49.349256 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215061db-a428-49e7-8943-a907537538a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:49.350492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215061db-a428-49e7-8943-a907537538a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:49.350954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215061db-a428-49e7-8943-a907537538a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:49.353224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215061db-a428-49e7-8943-a907537538a5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:49.371265 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 243/480] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:50.437769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a714fd0-c8b8-4162-a0f1-72cfbc3bbdc0 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:50.438272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a714fd0-c8b8-4162-a0f1-72cfbc3bbdc0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:50.438345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a714fd0-c8b8-4162-a0f1-72cfbc3bbdc0 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:50.439908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 238/481] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:50.445615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faedc1b-ed83-48f4-97cd-bf916cde7fc2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:50.446163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faedc1b-ed83-48f4-97cd-bf916cde7fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:50.446239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faedc1b-ed83-48f4-97cd-bf916cde7fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:50.474816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9faedc1b-ed83-48f4-97cd-bf916cde7fc2 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:50.476024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9faedc1b-ed83-48f4-97cd-bf916cde7fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:50.574893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 244/482] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:50.615255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a714fd0-c8b8-4162-a0f1-72cfbc3bbdc0 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:50.757658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a714fd0-c8b8-4162-a0f1-72cfbc3bbdc0 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:50.794715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb1f29-4ee8-4408-9692-008c932214e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:50.795656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1f29-4ee8-4408-9692-008c932214e6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:50.795934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1f29-4ee8-4408-9692-008c932214e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:50.796201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1f29-4ee8-4408-9692-008c932214e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:50.807060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb1f29-4ee8-4408-9692-008c932214e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-67bb1f29-4ee8-4408-9692-008c932214e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:50.815490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:50.816489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:50.817264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:50.817264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:50.817264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:50.817264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:50.817264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:50.817264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:50.817264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:50.817264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:50.817264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:50.817264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:50.817264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:03:50.817264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:50.820635 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 239/483] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:03:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:50.823048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faedc1b-ed83-48f4-97cd-bf916cde7fc2 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:50.912134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:50.931992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faedc1b-ed83-48f4-97cd-bf916cde7fc2 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:50.978033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038110ad-a9d4-4104-b535-02aab0d22877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:50.978766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038110ad-a9d4-4104-b535-02aab0d22877 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:50.979048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038110ad-a9d4-4104-b535-02aab0d22877 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:50.979317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038110ad-a9d4-4104-b535-02aab0d22877 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:50.980073 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038110ad-a9d4-4104-b535-02aab0d22877 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:50.980437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038110ad-a9d4-4104-b535-02aab0d22877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:50.982841 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038110ad-a9d4-4104-b535-02aab0d22877 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:51.000432 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 245/484] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:03:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:51.005236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb1f29-4ee8-4408-9692-008c932214e6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:51.050254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:51.154214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb1f29-4ee8-4408-9692-008c932214e6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:51.207479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15df1f-2ddf-46f3-9ca8-9e9944072ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:51.208209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15df1f-2ddf-46f3-9ca8-9e9944072ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:51.208491 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15df1f-2ddf-46f3-9ca8-9e9944072ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:51.208766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15df1f-2ddf-46f3-9ca8-9e9944072ea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:51.221219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15df1f-2ddf-46f3-9ca8-9e9944072ea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:51.227398 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15df1f-2ddf-46f3-9ca8-9e9944072ea1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-1f15df1f-2ddf-46f3-9ca8-9e9944072ea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:51.252569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:51.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:51.255318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:51.255318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:51.255318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:51.255318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:51.255318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 240/485] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:51.263544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038110ad-a9d4-4104-b535-02aab0d22877 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:51.291067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d5241-3d8e-45ea-a29d-4ad664d5491b None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:51.291454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d5241-3d8e-45ea-a29d-4ad664d5491b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:51.291666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d5241-3d8e-45ea-a29d-4ad664d5491b None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:51.293258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 241/486] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:03:51.301805 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432400fb-e433-4c07-add6-f7e8dd5b6f1d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:51.302140 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432400fb-e433-4c07-add6-f7e8dd5b6f1d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:51.302385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432400fb-e433-4c07-add6-f7e8dd5b6f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:51.328748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-432400fb-e433-4c07-add6-f7e8dd5b6f1d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:51.329973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-432400fb-e433-4c07-add6-f7e8dd5b6f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:51.483099 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 242/487] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:51.488376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038110ad-a9d4-4104-b535-02aab0d22877 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:51.527528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b185a8f-284a-44cb-a669-d0d568d7d563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:51.528805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b185a8f-284a-44cb-a669-d0d568d7d563 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:51.529134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b185a8f-284a-44cb-a669-d0d568d7d563 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:51.529416 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b185a8f-284a-44cb-a669-d0d568d7d563 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:51.530190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b185a8f-284a-44cb-a669-d0d568d7d563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:51.530500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b185a8f-284a-44cb-a669-d0d568d7d563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:51.531959 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b185a8f-284a-44cb-a669-d0d568d7d563 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:51.534120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432400fb-e433-4c07-add6-f7e8dd5b6f1d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:51.550137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:51.568535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 246/488] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:51.572471 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b185a8f-284a-44cb-a669-d0d568d7d563 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:51.594136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:51.748439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432400fb-e433-4c07-add6-f7e8dd5b6f1d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:51.792017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b185a8f-284a-44cb-a669-d0d568d7d563 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:51.823960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c7651-e349-4a1b-b8d4-e054bb09cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:51.825413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c7651-e349-4a1b-b8d4-e054bb09cf19 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:51.825916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c7651-e349-4a1b-b8d4-e054bb09cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:51.826349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c7651-e349-4a1b-b8d4-e054bb09cf19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:51.845545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c7651-e349-4a1b-b8d4-e054bb09cf19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-c74c7651-e349-4a1b-b8d4-e054bb09cf19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:51.853258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:51.854712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:51.856073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 243/489] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:03:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:51.862833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c7651-e349-4a1b-b8d4-e054bb09cf19 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:51.879421 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b634f-3309-42b1-b400-5e2937048584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:51.880337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b634f-3309-42b1-b400-5e2937048584 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:51.880617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b634f-3309-42b1-b400-5e2937048584 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:51.880891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b634f-3309-42b1-b400-5e2937048584 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:51.891440 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b634f-3309-42b1-b400-5e2937048584 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:51.893062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b634f-3309-42b1-b400-5e2937048584 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-943b634f-3309-42b1-b400-5e2937048584 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:03:51.926700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:51.927840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:51.928735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:51.928735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:51.928735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:03:51.928735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:03:51.928735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:03:51.928735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:03:51.928735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:03:51.928735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:03:51.928735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 18:03:51.928735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:51.933549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 247/490] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:51.933549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df0629-d8e1-4fd1-abf9-90233224b0bb None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:51.933549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df0629-d8e1-4fd1-abf9-90233224b0bb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:51.933549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df0629-d8e1-4fd1-abf9-90233224b0bb None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:51.934423 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 248/491] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:51.935517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df0629-d8e1-4fd1-abf9-90233224b0bb None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:51.980315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:52.023633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:52.023945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c7651-e349-4a1b-b8d4-e054bb09cf19 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:52.062382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cd3fa-fe75-4105-8b01-68188f880f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:52.063056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cd3fa-fe75-4105-8b01-68188f880f8b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.063365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cd3fa-fe75-4105-8b01-68188f880f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.063607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cd3fa-fe75-4105-8b01-68188f880f8b None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.064338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cd3fa-fe75-4105-8b01-68188f880f8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:52.064643 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cd3fa-fe75-4105-8b01-68188f880f8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:52.066161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cd3fa-fe75-4105-8b01-68188f880f8b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:52.077900 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 244/492] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:03:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:52.079915 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da7723-aada-41ff-8b3a-f0b7264d2b39 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.080340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da7723-aada-41ff-8b3a-f0b7264d2b39 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.080755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da7723-aada-41ff-8b3a-f0b7264d2b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.089157 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df0629-d8e1-4fd1-abf9-90233224b0bb None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:52.105905 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-66da7723-aada-41ff-8b3a-f0b7264d2b39 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:52.108240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-66da7723-aada-41ff-8b3a-f0b7264d2b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:52.129242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf81636-98b7-4f9e-8239-fcc7dde2f174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:52.130413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf81636-98b7-4f9e-8239-fcc7dde2f174 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.130875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf81636-98b7-4f9e-8239-fcc7dde2f174 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.131303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf81636-98b7-4f9e-8239-fcc7dde2f174 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.132284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf81636-98b7-4f9e-8239-fcc7dde2f174 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:52.132776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf81636-98b7-4f9e-8239-fcc7dde2f174 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:52.134681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf81636-98b7-4f9e-8239-fcc7dde2f174 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:52.183787 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 249/493] 10.222.0.46 () {56 vars in 1137 bytes} [Thu May 26 18:03:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:52.186999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77348bf9-eadf-4267-b0f9-73700e8940c0 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.187287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77348bf9-eadf-4267-b0f9-73700e8940c0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.187582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77348bf9-eadf-4267-b0f9-73700e8940c0 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.189391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 250/494] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:52.190683 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77348bf9-eadf-4267-b0f9-73700e8940c0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:52.214863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 245/495] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:52.215849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da7723-aada-41ff-8b3a-f0b7264d2b39 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:52.288240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:52.350448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da7723-aada-41ff-8b3a-f0b7264d2b39 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:52.380981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77348bf9-eadf-4267-b0f9-73700e8940c0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:52.407736 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bb6dd-1b9c-46fb-8127-75d33127a15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:52.408915 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bb6dd-1b9c-46fb-8127-75d33127a15b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.409402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bb6dd-1b9c-46fb-8127-75d33127a15b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.409871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bb6dd-1b9c-46fb-8127-75d33127a15b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.418983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:52.435505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244b9c1-ae5e-4531-b857-13144ef8194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:52.436586 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244b9c1-ae5e-4531-b857-13144ef8194f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.437075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244b9c1-ae5e-4531-b857-13144ef8194f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.437523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244b9c1-ae5e-4531-b857-13144ef8194f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.450815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244b9c1-ae5e-4531-b857-13144ef8194f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:52.452442 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244b9c1-ae5e-4531-b857-13144ef8194f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-0244b9c1-ae5e-4531-b857-13144ef8194f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:52.472030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:52.473238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:52.474885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:52.474885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:52.474885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:52.474885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:52.474885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 251/496] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:52.474885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb25c3-087a-4879-9aa1-5988017f43fc None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.475503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb25c3-087a-4879-9aa1-5988017f43fc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.475503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb25c3-087a-4879-9aa1-5988017f43fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.494911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ddfb25c3-087a-4879-9aa1-5988017f43fc None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:52.497097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42bb6dd-1b9c-46fb-8127-75d33127a15b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:52.497685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ddfb25c3-087a-4879-9aa1-5988017f43fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:52.498629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42bb6dd-1b9c-46fb-8127-75d33127a15b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:52.560434 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:52.572875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 252/497] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:52.573980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb25c3-087a-4879-9aa1-5988017f43fc None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:52.606986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 246/498] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:52.608322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62f93a-9b90-4652-81c7-c3f54aa47ab5 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.608607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62f93a-9b90-4652-81c7-c3f54aa47ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.608883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62f93a-9b90-4652-81c7-c3f54aa47ab5 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.610191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 247/499] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:52.610888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62f93a-9b90-4652-81c7-c3f54aa47ab5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:52.686215 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb25c3-087a-4879-9aa1-5988017f43fc None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:52.689287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:52.725660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62f93a-9b90-4652-81c7-c3f54aa47ab5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:52.727440 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d769718-924e-4db2-aa44-554bab6cc7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:52.728241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d769718-924e-4db2-aa44-554bab6cc7bd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.728518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d769718-924e-4db2-aa44-554bab6cc7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.728790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d769718-924e-4db2-aa44-554bab6cc7bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.739727 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d769718-924e-4db2-aa44-554bab6cc7bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-4d769718-924e-4db2-aa44-554bab6cc7bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:52.748437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:52.750043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 253/500] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:03:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:52.751582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d769718-924e-4db2-aa44-554bab6cc7bd None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:52.769153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb8e8f-f67f-42ef-b6eb-13ed873c23b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:52.769842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb8e8f-f67f-42ef-b6eb-13ed873c23b5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.770122 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb8e8f-f67f-42ef-b6eb-13ed873c23b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.770401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb8e8f-f67f-42ef-b6eb-13ed873c23b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.771118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb8e8f-f67f-42ef-b6eb-13ed873c23b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:52.771423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb8e8f-f67f-42ef-b6eb-13ed873c23b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:52.772797 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb8e8f-f67f-42ef-b6eb-13ed873c23b5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:52.784722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 248/501] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:52.785827 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb8e8f-f67f-42ef-b6eb-13ed873c23b5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:52.816745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:52.864404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d769718-924e-4db2-aa44-554bab6cc7bd None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:52.899976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3503f86-19fe-448e-9f86-e454d75fb3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:52.900867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb8e8f-f67f-42ef-b6eb-13ed873c23b5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:52.900990 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3503f86-19fe-448e-9f86-e454d75fb3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.901396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3503f86-19fe-448e-9f86-e454d75fb3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.901764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3503f86-19fe-448e-9f86-e454d75fb3d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.913661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3503f86-19fe-448e-9f86-e454d75fb3d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f3503f86-19fe-448e-9f86-e454d75fb3d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:52.921985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:52.923708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:52.925274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:52.925274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:52.925274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:52.925274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:52.925274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:52.925274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:52.925274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:52.925274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:52.925274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:52.925274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:52.925274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:03:52.925274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:52.928162 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 254/502] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:03:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:52.929456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1aac25-8e44-4541-9ac6-02a96fa863f5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.929754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1aac25-8e44-4541-9ac6-02a96fa863f5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.930044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1aac25-8e44-4541-9ac6-02a96fa863f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.942809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457b24c-1e32-43f5-a953-a849f94963d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:52.943605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457b24c-1e32-43f5-a953-a849f94963d4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:52.943883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457b24c-1e32-43f5-a953-a849f94963d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:52.944156 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457b24c-1e32-43f5-a953-a849f94963d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:52.947505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5a1aac25-8e44-4541-9ac6-02a96fa863f5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:52.948720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5a1aac25-8e44-4541-9ac6-02a96fa863f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:52.956120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457b24c-1e32-43f5-a953-a849f94963d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:52.957585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457b24c-1e32-43f5-a953-a849f94963d4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:52.968290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e457b24c-1e32-43f5-a953-a849f94963d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:52.975646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:52.977163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:52.978930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:52.978930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:52.978930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:52.978930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:52.978930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 249/503] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:52.978930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457b24c-1e32-43f5-a953-a849f94963d4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:53.027485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 255/504] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:53.029245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1aac25-8e44-4541-9ac6-02a96fa863f5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:53.072862 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:53.116087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457b24c-1e32-43f5-a953-a849f94963d4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:53.136003 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1aac25-8e44-4541-9ac6-02a96fa863f5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:53.157264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ddb81-e5ba-4438-a1fb-c834127398ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:53.157959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ddb81-e5ba-4438-a1fb-c834127398ac None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:53.158240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ddb81-e5ba-4438-a1fb-c834127398ac None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:53.158512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ddb81-e5ba-4438-a1fb-c834127398ac None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:53.159270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ddb81-e5ba-4438-a1fb-c834127398ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:53.159558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ddb81-e5ba-4438-a1fb-c834127398ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:53.161181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ddb81-e5ba-4438-a1fb-c834127398ac None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:53.170136 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 250/505] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:03:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:53.171922 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ddb81-e5ba-4438-a1fb-c834127398ac None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:53.180408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee3fb0-4b0c-4a6e-a538-3e44c129782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:53.181389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee3fb0-4b0c-4a6e-a538-3e44c129782c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:53.181666 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee3fb0-4b0c-4a6e-a538-3e44c129782c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:53.181969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee3fb0-4b0c-4a6e-a538-3e44c129782c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:53.192035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee3fb0-4b0c-4a6e-a538-3e44c129782c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:53.193532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee3fb0-4b0c-4a6e-a538-3e44c129782c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-bfee3fb0-4b0c-4a6e-a538-3e44c129782c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:03:53.211236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:53.212647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:53.214764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:53.214764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:53.214764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:03:53.214764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:03:53.214764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:03:53.214764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:03:53.214764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:03:53.214764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:03:53.214764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 18:03:53.214764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:53.214764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 256/506] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:53.214764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee3fb0-4b0c-4a6e-a538-3e44c129782c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:53.229678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:53.286838 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ddb81-e5ba-4438-a1fb-c834127398ac None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:53.323543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee3fb0-4b0c-4a6e-a538-3e44c129782c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:53.323699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468abec6-b65d-4a1c-b2c7-44aa26b01e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:53.324192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468abec6-b65d-4a1c-b2c7-44aa26b01e45 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:53.324474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468abec6-b65d-4a1c-b2c7-44aa26b01e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:53.325287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468abec6-b65d-4a1c-b2c7-44aa26b01e45 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:53.326214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468abec6-b65d-4a1c-b2c7-44aa26b01e45 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:53.326518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468abec6-b65d-4a1c-b2c7-44aa26b01e45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:53.327894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468abec6-b65d-4a1c-b2c7-44aa26b01e45 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:53.329460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:53.337396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 251/507] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:03:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:53.338243 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468abec6-b65d-4a1c-b2c7-44aa26b01e45 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:53.370081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcd921-8760-453d-8c4e-76435a8cd8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:53.370812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcd921-8760-453d-8c4e-76435a8cd8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:53.371160 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcd921-8760-453d-8c4e-76435a8cd8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:53.372098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcd921-8760-453d-8c4e-76435a8cd8f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:53.372900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdcd921-8760-453d-8c4e-76435a8cd8f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:53.373403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdcd921-8760-453d-8c4e-76435a8cd8f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:53.375128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdcd921-8760-453d-8c4e-76435a8cd8f2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:53.387777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 257/508] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:53.388604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcd921-8760-453d-8c4e-76435a8cd8f2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:53.453046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468abec6-b65d-4a1c-b2c7-44aa26b01e45 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:53.462891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:53.488101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcb2bc-4973-4682-8694-840499bac0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:53.488880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcd921-8760-453d-8c4e-76435a8cd8f2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:53.489019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcb2bc-4973-4682-8694-840499bac0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:53.489161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcb2bc-4973-4682-8694-840499bac0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:53.489417 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcb2bc-4973-4682-8694-840499bac0e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:53.503194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fcb2bc-4973-4682-8694-840499bac0e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:53.505923 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fcb2bc-4973-4682-8694-840499bac0e1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:53.526885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e077f9-5698-4c62-8e30-7aef10470e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:53.528198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e077f9-5698-4c62-8e30-7aef10470e33 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:53.528556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e077f9-5698-4c62-8e30-7aef10470e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-37fcb2bc-4973-4682-8694-840499bac0e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:53.528723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:53.529664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:53.531430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:53.531430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:53.531430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:53.531430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:53.531851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e077f9-5698-4c62-8e30-7aef10470e33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:53.531977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 252/509] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:53.531977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcb2bc-4973-4682-8694-840499bac0e1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:53.548650 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e077f9-5698-4c62-8e30-7aef10470e33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:53.550161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e077f9-5698-4c62-8e30-7aef10470e33 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-08e077f9-5698-4c62-8e30-7aef10470e33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:53.570603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:53.572500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:53.574536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:53.574536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:53.574536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:53.574536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:53.574536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 258/510] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:53.574536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e077f9-5698-4c62-8e30-7aef10470e33 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:53.591619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:53.646756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcb2bc-4973-4682-8694-840499bac0e1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:53.682394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999cf56-b07c-4df0-a98d-ee5cf70c854a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:53.682882 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e077f9-5698-4c62-8e30-7aef10470e33 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:53.683053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999cf56-b07c-4df0-a98d-ee5cf70c854a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:53.683330 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999cf56-b07c-4df0-a98d-ee5cf70c854a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:53.683598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999cf56-b07c-4df0-a98d-ee5cf70c854a None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:53.685157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999cf56-b07c-4df0-a98d-ee5cf70c854a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:53.685463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999cf56-b07c-4df0-a98d-ee5cf70c854a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:53.686924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999cf56-b07c-4df0-a98d-ee5cf70c854a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:53.709600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:53.719634 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854575fe-651b-4538-93d9-33e43f6be237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:53.720425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854575fe-651b-4538-93d9-33e43f6be237 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:53.720702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854575fe-651b-4538-93d9-33e43f6be237 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:53.720969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854575fe-651b-4538-93d9-33e43f6be237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:53.726837 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 253/511] 10.222.0.46 () {56 vars in 1137 bytes} [Thu May 26 18:03:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:53.727664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999cf56-b07c-4df0-a98d-ee5cf70c854a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:53.737354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854575fe-651b-4538-93d9-33e43f6be237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:53.739249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854575fe-651b-4538-93d9-33e43f6be237 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-854575fe-651b-4538-93d9-33e43f6be237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:53.756999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:53.758244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:53.759823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:53.759823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:53.759823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:53.759823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:53.759823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 259/512] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:53.759823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854575fe-651b-4538-93d9-33e43f6be237 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:53.845001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999cf56-b07c-4df0-a98d-ee5cf70c854a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:53.852575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:53.867830 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854575fe-651b-4538-93d9-33e43f6be237 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:53.881180 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c31c05-6165-40c0-ac60-4f7af03184c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:53.882000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c31c05-6165-40c0-ac60-4f7af03184c2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:53.882280 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c31c05-6165-40c0-ac60-4f7af03184c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:53.882552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c31c05-6165-40c0-ac60-4f7af03184c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:53.890302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c31c05-6165-40c0-ac60-4f7af03184c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:53.891776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c31c05-6165-40c0-ac60-4f7af03184c2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:53.906732 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d8a09-ac00-464e-912d-343df1765837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-36c31c05-6165-40c0-ac60-4f7af03184c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:03:53.906951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:53.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:53.910974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:03:53.910974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:03:53.910974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:03:53.910974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:03:53.910974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:03:53.910974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:03:53.910974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:03:53.910974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:53.911870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d8a09-ac00-464e-912d-343df1765837 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:53.911870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d8a09-ac00-464e-912d-343df1765837 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:53.911870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d8a09-ac00-464e-912d-343df1765837 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:53.912318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 254/513] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:53.912471 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d8a09-ac00-464e-912d-343df1765837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:53.912471 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d8a09-ac00-464e-912d-343df1765837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:53.912708 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c31c05-6165-40c0-ac60-4f7af03184c2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:53.912843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d8a09-ac00-464e-912d-343df1765837 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:53.925795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 260/514] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:53.926506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d8a09-ac00-464e-912d-343df1765837 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:53.973008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:54.029507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c31c05-6165-40c0-ac60-4f7af03184c2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:54.037167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d8a09-ac00-464e-912d-343df1765837 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:54.065701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5aaa41-0d89-4789-9155-46d3ed8cf6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:54.066639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5aaa41-0d89-4789-9155-46d3ed8cf6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:54.067042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5aaa41-0d89-4789-9155-46d3ed8cf6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:54.067497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5aaa41-0d89-4789-9155-46d3ed8cf6b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:54.068388 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5aaa41-0d89-4789-9155-46d3ed8cf6b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:54.068771 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5aaa41-0d89-4789-9155-46d3ed8cf6b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:54.070391 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5aaa41-0d89-4789-9155-46d3ed8cf6b5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:54.077164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25884960-e488-46b9-9fa6-605c6f816297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:54.077840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25884960-e488-46b9-9fa6-605c6f816297 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:54.078120 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25884960-e488-46b9-9fa6-605c6f816297 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:54.078389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25884960-e488-46b9-9fa6-605c6f816297 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:54.084727 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 255/515] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:54.085566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5aaa41-0d89-4789-9155-46d3ed8cf6b5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:54.116094 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:54.152526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25884960-e488-46b9-9fa6-605c6f816297 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:54.154072 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25884960-e488-46b9-9fa6-605c6f816297 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:54.195404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:54.204575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5aaa41-0d89-4789-9155-46d3ed8cf6b5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:54.239920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a459d8-be34-49a3-8419-7d2ce4ece14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:54.240588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a459d8-be34-49a3-8419-7d2ce4ece14f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:54.240863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a459d8-be34-49a3-8419-7d2ce4ece14f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:54.241137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a459d8-be34-49a3-8419-7d2ce4ece14f None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:54.241889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a459d8-be34-49a3-8419-7d2ce4ece14f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:54.242194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a459d8-be34-49a3-8419-7d2ce4ece14f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:54.243566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a459d8-be34-49a3-8419-7d2ce4ece14f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:54.255341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 256/516] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:54.256176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a459d8-be34-49a3-8419-7d2ce4ece14f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:54.263687 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 261/517] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:54.264515 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25884960-e488-46b9-9fa6-605c6f816297 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:54.359121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:54.375538 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a459d8-be34-49a3-8419-7d2ce4ece14f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:54.378704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25884960-e488-46b9-9fa6-605c6f816297 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:54.411463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e00505-c744-40d2-932e-c5ee44aa1ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:54.412120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e00505-c744-40d2-932e-c5ee44aa1ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:54.412404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e00505-c744-40d2-932e-c5ee44aa1ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:54.412675 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e00505-c744-40d2-932e-c5ee44aa1ea2 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:54.413449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e00505-c744-40d2-932e-c5ee44aa1ea2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:54.413779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e00505-c744-40d2-932e-c5ee44aa1ea2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:54.415222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e00505-c744-40d2-932e-c5ee44aa1ea2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:54.415318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2acf187-9685-40e4-a352-751696b838c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:54.416001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acf187-9685-40e4-a352-751696b838c7 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:54.416342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acf187-9685-40e4-a352-751696b838c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:54.416814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acf187-9685-40e4-a352-751696b838c7 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:54.417519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2acf187-9685-40e4-a352-751696b838c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:54.419563 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2acf187-9685-40e4-a352-751696b838c7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:54.428932 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:54.435948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 257/518] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:03:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:54.437008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e00505-c744-40d2-932e-c5ee44aa1ea2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:54.446813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 262/519] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:03:54 2022] POST /identity/v3/users => generated 247 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:54.448056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2acf187-9685-40e4-a352-751696b838c7 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:54.577919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e00505-c744-40d2-932e-c5ee44aa1ea2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:54.611183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2acf187-9685-40e4-a352-751696b838c7 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:54.637392 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564b8d1-376b-43ef-8a57-e49dd89c3657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:54.638558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564b8d1-376b-43ef-8a57-e49dd89c3657 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:54.638886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564b8d1-376b-43ef-8a57-e49dd89c3657 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:54.639311 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564b8d1-376b-43ef-8a57-e49dd89c3657 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:54.649970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564b8d1-376b-43ef-8a57-e49dd89c3657 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:54.651447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564b8d1-376b-43ef-8a57-e49dd89c3657 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:54.669382 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9776256-c29b-413c-b9dd-09a51c3d76e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-7564b8d1-376b-43ef-8a57-e49dd89c3657 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:03:54.670118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:54.671167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:54.672881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:54.672881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:54.672881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:03:54.672881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:03:54.672881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:03:54.672881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:03:54.672881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:03:54.672881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:03:54.672881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 18:03:54.672881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:54.674244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9776256-c29b-413c-b9dd-09a51c3d76e5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:54.674244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9776256-c29b-413c-b9dd-09a51c3d76e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:54.674244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9776256-c29b-413c-b9dd-09a51c3d76e5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:54.674644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 258/520] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 18:03:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:54.674644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564b8d1-376b-43ef-8a57-e49dd89c3657 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:54.682764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9776256-c29b-413c-b9dd-09a51c3d76e5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:54.685447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9776256-c29b-413c-b9dd-09a51c3d76e5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:54.690824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f9776256-c29b-413c-b9dd-09a51c3d76e5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:03:54.705323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:54.707197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:54.709116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:54.709116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:54.709116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:03:54.709116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:03:54.709116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:03:54.709116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:03:54.709116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:03:54.709116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:03:54.709116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 18:03:54.709116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:54.709116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 263/521] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:03:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:54.709116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9776256-c29b-413c-b9dd-09a51c3d76e5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:54.773536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:54.818301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564b8d1-376b-43ef-8a57-e49dd89c3657 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:54.839584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9776256-c29b-413c-b9dd-09a51c3d76e5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:54.874256 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc5681-68ca-42f7-bbff-7490ad8e354f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:54.875704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc5681-68ca-42f7-bbff-7490ad8e354f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:54.875958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc5681-68ca-42f7-bbff-7490ad8e354f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:54.876236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc5681-68ca-42f7-bbff-7490ad8e354f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:54.882671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d9889-284c-4e62-8dd5-73dbed4a75eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:54.883487 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d9889-284c-4e62-8dd5-73dbed4a75eb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:54.883912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d9889-284c-4e62-8dd5-73dbed4a75eb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:54.884314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d9889-284c-4e62-8dd5-73dbed4a75eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:54.885306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d9889-284c-4e62-8dd5-73dbed4a75eb None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=8c79367697f7451fb804b8641444f6aa) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:54.885798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d9889-284c-4e62-8dd5-73dbed4a75eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:54.888069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d9889-284c-4e62-8dd5-73dbed4a75eb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:54.895494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc5681-68ca-42f7-bbff-7490ad8e354f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:54.897873 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc5681-68ca-42f7-bbff-7490ad8e354f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:54.902780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 264/522] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:54 2022] GET /identity/v3/role_assignments?user.id=8c79367697f7451fb804b8641444f6aa&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:54.904689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d9889-284c-4e62-8dd5-73dbed4a75eb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-02bc5681-68ca-42f7-bbff-7490ad8e354f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:54.923190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:54.924686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:54.926360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:54.926360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:54.926360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:54.926360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:54.929230 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 259/523] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:54.930243 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc5681-68ca-42f7-bbff-7490ad8e354f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:55.017703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:55.068697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc5681-68ca-42f7-bbff-7490ad8e354f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:55.096609 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d9889-284c-4e62-8dd5-73dbed4a75eb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:55.108624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b6d9-655c-49a1-94c4-5371b5ee5dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:55.109315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b6d9-655c-49a1-94c4-5371b5ee5dea None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:55.109596 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b6d9-655c-49a1-94c4-5371b5ee5dea None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:55.111811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b6d9-655c-49a1-94c4-5371b5ee5dea None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:55.112724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750b6d9-655c-49a1-94c4-5371b5ee5dea None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:55.113041 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750b6d9-655c-49a1-94c4-5371b5ee5dea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:55.115061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750b6d9-655c-49a1-94c4-5371b5ee5dea None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:55.134100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:55.142845 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 260/524] 10.222.0.46 () {56 vars in 1139 bytes} [Thu May 26 18:03:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:55.143748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b6d9-655c-49a1-94c4-5371b5ee5dea None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:55.168822 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7e498-701b-4f3b-a20c-23d86dbe74ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:55.170371 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7e498-701b-4f3b-a20c-23d86dbe74ac None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:55.171079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7e498-701b-4f3b-a20c-23d86dbe74ac None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:55.171743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7e498-701b-4f3b-a20c-23d86dbe74ac None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:55.173316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7e498-701b-4f3b-a20c-23d86dbe74ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:55.174094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7e498-701b-4f3b-a20c-23d86dbe74ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:55.176542 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7e498-701b-4f3b-a20c-23d86dbe74ac None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:55.207086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 265/525] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 18:03:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:55.208653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7e498-701b-4f3b-a20c-23d86dbe74ac None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:55.314309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b6d9-655c-49a1-94c4-5371b5ee5dea None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:55.358173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:55.384277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7e498-701b-4f3b-a20c-23d86dbe74ac None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:55.396593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d4b60-af44-4254-9f58-8bb8d27acd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:55.398067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d4b60-af44-4254-9f58-8bb8d27acd7a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:55.398762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d4b60-af44-4254-9f58-8bb8d27acd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:55.399468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d4b60-af44-4254-9f58-8bb8d27acd7a None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:55.401051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d4b60-af44-4254-9f58-8bb8d27acd7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:55.401841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d4b60-af44-4254-9f58-8bb8d27acd7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:55.405427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d4b60-af44-4254-9f58-8bb8d27acd7a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:55.423322 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7800927-7165-40aa-bcc5-5099d3dbc230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:55.424468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7800927-7165-40aa-bcc5-5099d3dbc230 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:55.424468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7800927-7165-40aa-bcc5-5099d3dbc230 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:55.424784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7800927-7165-40aa-bcc5-5099d3dbc230 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:55.430943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 261/526] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:55.432590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d4b60-af44-4254-9f58-8bb8d27acd7a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:55.480064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:55.506805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7800927-7165-40aa-bcc5-5099d3dbc230 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:55.508356 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7800927-7165-40aa-bcc5-5099d3dbc230 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:55.622002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d4b60-af44-4254-9f58-8bb8d27acd7a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:55.634776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 266/527] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:55.635548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7800927-7165-40aa-bcc5-5099d3dbc230 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:55.690656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc789a97-6b30-46a4-8c35-9d2b91e66d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:55.692101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc789a97-6b30-46a4-8c35-9d2b91e66d78 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:55.692814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc789a97-6b30-46a4-8c35-9d2b91e66d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:55.693533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc789a97-6b30-46a4-8c35-9d2b91e66d78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:55.726288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:55.753384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7800927-7165-40aa-bcc5-5099d3dbc230 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:55.785781 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:55.793306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9bf8a-9e7a-44eb-bafc-16ce58b2f07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:55.794251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9bf8a-9e7a-44eb-bafc-16ce58b2f07d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:55.794879 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9bf8a-9e7a-44eb-bafc-16ce58b2f07d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:55.795003 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9bf8a-9e7a-44eb-bafc-16ce58b2f07d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:55.802997 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9bf8a-9e7a-44eb-bafc-16ce58b2f07d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:55.804709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9bf8a-9e7a-44eb-bafc-16ce58b2f07d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:55.811806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc789a97-6b30-46a4-8c35-9d2b91e66d78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:55.813461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc789a97-6b30-46a4-8c35-9d2b91e66d78 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-88a9bf8a-9e7a-44eb-bafc-16ce58b2f07d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:03:55.821000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:55.822065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:55.822985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:03:55.822985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:03:55.822985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:03:55.822985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:03:55.822985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:03:55.822985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:03:55.822985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:03:55.822985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:55.826097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 267/528] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:55.827206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9bf8a-9e7a-44eb-bafc-16ce58b2f07d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:55.941890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9bf8a-9e7a-44eb-bafc-16ce58b2f07d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:55.942213 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 262/529] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:03:55.943709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc789a97-6b30-46a4-8c35-9d2b91e66d78 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:55.986099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9430e-9974-4953-af50-20a902e91dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:55.986890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9430e-9974-4953-af50-20a902e91dcb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:55.987194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9430e-9974-4953-af50-20a902e91dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:55.987601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9430e-9974-4953-af50-20a902e91dcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:55.990662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:56.001666 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9430e-9974-4953-af50-20a902e91dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:56.003203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9430e-9974-4953-af50-20a902e91dcb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-aad9430e-9974-4953-af50-20a902e91dcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:56.023136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:56.024811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:56.026341 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:56.026341 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:56.026341 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:56.026341 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:56.029785 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 268/530] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:56.030693 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9430e-9974-4953-af50-20a902e91dcb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:56.070334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:56.105516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc789a97-6b30-46a4-8c35-9d2b91e66d78 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:56.144277 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02077e-b388-47c8-b3a0-52077f057915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:56.145215 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02077e-b388-47c8-b3a0-52077f057915 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:56.145652 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02077e-b388-47c8-b3a0-52077f057915 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:56.146119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02077e-b388-47c8-b3a0-52077f057915 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:56.146231 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9430e-9974-4953-af50-20a902e91dcb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:56.147152 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02077e-b388-47c8-b3a0-52077f057915 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:56.147612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02077e-b388-47c8-b3a0-52077f057915 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:56.149178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02077e-b388-47c8-b3a0-52077f057915 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:56.162772 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 263/531] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:03:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:56.163926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02077e-b388-47c8-b3a0-52077f057915 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:56.191710 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80efda2-6c29-4967-9b3a-935b92605649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:56.192556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80efda2-6c29-4967-9b3a-935b92605649 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:56.192789 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80efda2-6c29-4967-9b3a-935b92605649 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:56.193060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80efda2-6c29-4967-9b3a-935b92605649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:56.211029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80efda2-6c29-4967-9b3a-935b92605649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:56.212872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80efda2-6c29-4967-9b3a-935b92605649 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c80efda2-6c29-4967-9b3a-935b92605649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:56.235979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:56.236877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:56.237697 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:56.237697 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:56.237697 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:56.237697 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:56.243667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 269/532] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:56.243667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80efda2-6c29-4967-9b3a-935b92605649 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:56.291628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02077e-b388-47c8-b3a0-52077f057915 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:56.295936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:56.329683 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a6b8e-7035-4289-b5b4-3abdeb109683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:56.330622 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a6b8e-7035-4289-b5b4-3abdeb109683 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:56.331119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a6b8e-7035-4289-b5b4-3abdeb109683 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:56.331596 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a6b8e-7035-4289-b5b4-3abdeb109683 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:56.332536 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a6b8e-7035-4289-b5b4-3abdeb109683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:56.333046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a6b8e-7035-4289-b5b4-3abdeb109683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:56.334650 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a6b8e-7035-4289-b5b4-3abdeb109683 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:56.337569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:56.352571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 264/533] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:56.353942 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a6b8e-7035-4289-b5b4-3abdeb109683 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:56.381516 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80efda2-6c29-4967-9b3a-935b92605649 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:56.421291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e192ceb-619c-44c3-a434-d9d36ded8ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:56.421975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e192ceb-619c-44c3-a434-d9d36ded8ccc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:56.422258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e192ceb-619c-44c3-a434-d9d36ded8ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:56.422532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e192ceb-619c-44c3-a434-d9d36ded8ccc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:56.423415 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e192ceb-619c-44c3-a434-d9d36ded8ccc None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=4a67120485f74f8bb0ff188cd3abd21c) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:56.423784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e192ceb-619c-44c3-a434-d9d36ded8ccc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:56.425525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e192ceb-619c-44c3-a434-d9d36ded8ccc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:56.438189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 270/534] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:56 2022] GET /identity/v3/role_assignments?user.id=4a67120485f74f8bb0ff188cd3abd21c&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:56.439353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e192ceb-619c-44c3-a434-d9d36ded8ccc None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:56.462834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a6b8e-7035-4289-b5b4-3abdeb109683 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:56.500990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2cbfd-4634-4535-900d-13d1903f35eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:56.502534 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2cbfd-4634-4535-900d-13d1903f35eb None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:56.503278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2cbfd-4634-4535-900d-13d1903f35eb None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:56.504044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2cbfd-4634-4535-900d-13d1903f35eb None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:56.505656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2cbfd-4634-4535-900d-13d1903f35eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:56.506496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2cbfd-4634-4535-900d-13d1903f35eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:56.508639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2cbfd-4634-4535-900d-13d1903f35eb None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:56.526979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 265/535] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:56.529504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2cbfd-4634-4535-900d-13d1903f35eb None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:56.592915 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:56.640301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e192ceb-619c-44c3-a434-d9d36ded8ccc None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:56.642909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:56.662933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2cbfd-4634-4535-900d-13d1903f35eb None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:56.683458 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2feb41-e03e-4386-91d6-58b3a2950f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:56.685258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2feb41-e03e-4386-91d6-58b3a2950f75 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:56.685802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2feb41-e03e-4386-91d6-58b3a2950f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:56.686294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2feb41-e03e-4386-91d6-58b3a2950f75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:56.693962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2feb41-e03e-4386-91d6-58b3a2950f75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:56.695797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2feb41-e03e-4386-91d6-58b3a2950f75 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-ca2feb41-e03e-4386-91d6-58b3a2950f75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:03:56.711424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:56.712412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:56.713287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:03:56.713287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:03:56.713287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:03:56.713287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:03:56.713287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:03:56.713287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:03:56.713287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:03:56.713287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:56.716228 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 271/536] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:56.724976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2feb41-e03e-4386-91d6-58b3a2950f75 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:56.734366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e3d57-8335-41f1-8f88-2bcf9092698a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:56.735173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e3d57-8335-41f1-8f88-2bcf9092698a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:56.735497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e3d57-8335-41f1-8f88-2bcf9092698a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:56.735795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e3d57-8335-41f1-8f88-2bcf9092698a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:56.752040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e3d57-8335-41f1-8f88-2bcf9092698a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:56.753511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e3d57-8335-41f1-8f88-2bcf9092698a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-401e3d57-8335-41f1-8f88-2bcf9092698a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:03:56.781461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:56.783176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:56.784814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:03:56.784814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:03:56.784814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:03:56.784814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:03:56.784814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:03:56.784814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:03:56.784814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:03:56.784814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:56.790002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 266/537] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:03:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:56.792082 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e3d57-8335-41f1-8f88-2bcf9092698a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:56.999358 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2feb41-e03e-4386-91d6-58b3a2950f75 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:57.018370 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e3d57-8335-41f1-8f88-2bcf9092698a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:57.050551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:57.071460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966faa2-78b1-400c-8302-f53b93358cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:57.072152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966faa2-78b1-400c-8302-f53b93358cce None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:57.072551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966faa2-78b1-400c-8302-f53b93358cce None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:57.072719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966faa2-78b1-400c-8302-f53b93358cce None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:57.073475 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966faa2-78b1-400c-8302-f53b93358cce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:57.073812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966faa2-78b1-400c-8302-f53b93358cce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:57.075422 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966faa2-78b1-400c-8302-f53b93358cce None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:57.098177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea73e71-76e0-435d-83c4-6a55c6f470d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:57.099771 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea73e71-76e0-435d-83c4-6a55c6f470d6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:57.100575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea73e71-76e0-435d-83c4-6a55c6f470d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:57.101343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea73e71-76e0-435d-83c4-6a55c6f470d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:57.103173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea73e71-76e0-435d-83c4-6a55c6f470d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:57.104027 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea73e71-76e0-435d-83c4-6a55c6f470d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:57.114192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea73e71-76e0-435d-83c4-6a55c6f470d6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:57.132601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 272/538] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:03:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:57.142975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966faa2-78b1-400c-8302-f53b93358cce None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:57.143195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 267/539] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:03:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:57.149066 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea73e71-76e0-435d-83c4-6a55c6f470d6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:57.194736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:57.366051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea73e71-76e0-435d-83c4-6a55c6f470d6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:57.373031 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966faa2-78b1-400c-8302-f53b93358cce None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:57.404604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905b19c-581d-4382-a1ad-81af8e987bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:57.405270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905b19c-581d-4382-a1ad-81af8e987bdd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:57.405545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905b19c-581d-4382-a1ad-81af8e987bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:57.405847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905b19c-581d-4382-a1ad-81af8e987bdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:57.406676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905b19c-581d-4382-a1ad-81af8e987bdd None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=d420e808730d484f80963322c5fc6856) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:57.406984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905b19c-581d-4382-a1ad-81af8e987bdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:57.408515 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905b19c-581d-4382-a1ad-81af8e987bdd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:57.415646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 268/540] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:57 2022] GET /identity/v3/role_assignments?user.id=d420e808730d484f80963322c5fc6856&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:57.446638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b0325-9244-4e06-93c7-ccd3575e18bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:57.447327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b0325-9244-4e06-93c7-ccd3575e18bd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:57.447612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b0325-9244-4e06-93c7-ccd3575e18bd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:57.447889 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b0325-9244-4e06-93c7-ccd3575e18bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:57.448754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b0325-9244-4e06-93c7-ccd3575e18bd None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=f668c4c6745e4f1eae20d8a3a12cf0cc) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:57.449069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b0325-9244-4e06-93c7-ccd3575e18bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:57.451226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b0325-9244-4e06-93c7-ccd3575e18bd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:57.464325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 273/541] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:03:57 2022] GET /identity/v3/role_assignments?user.id=f668c4c6745e4f1eae20d8a3a12cf0cc&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:57.489586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff8ff8-64bf-4947-a84c-9435043e538f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:57.489936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff8ff8-64bf-4947-a84c-9435043e538f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:57.490186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff8ff8-64bf-4947-a84c-9435043e538f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:57.492002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 269/542] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:57.498124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee756b-bc0f-4b21-9b71-e9b87aebd1be None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:57.498465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee756b-bc0f-4b21-9b71-e9b87aebd1be None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:57.498687 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee756b-bc0f-4b21-9b71-e9b87aebd1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:57.528629 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-93ee756b-bc0f-4b21-9b71-e9b87aebd1be None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:57.529927 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-93ee756b-bc0f-4b21-9b71-e9b87aebd1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:57.653826 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 274/543] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:57.699299 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff8ff8-64bf-4947-a84c-9435043e538f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:57.772868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:57.855004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff8ff8-64bf-4947-a84c-9435043e538f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:57.904938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bddf7-da5e-4888-ab02-911c7e60a321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:57.905656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bddf7-da5e-4888-ab02-911c7e60a321 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:57.905976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bddf7-da5e-4888-ab02-911c7e60a321 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:57.906247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bddf7-da5e-4888-ab02-911c7e60a321 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:57.920368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bddf7-da5e-4888-ab02-911c7e60a321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-433bddf7-da5e-4888-ab02-911c7e60a321 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:57.928718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:57.930990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:57.932929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 270/544] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:03:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:57.942456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee756b-bc0f-4b21-9b71-e9b87aebd1be None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:57.975987 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:58.091258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee756b-bc0f-4b21-9b71-e9b87aebd1be None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.115218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b537a4-1f43-44cd-89b6-ad0bbadf32f5 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.115539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b537a4-1f43-44cd-89b6-ad0bbadf32f5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.115771 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b537a4-1f43-44cd-89b6-ad0bbadf32f5 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.117384 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 271/545] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:03:58.124139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff201-c9e6-4650-827d-9b0f2f1e344d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.124438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff201-c9e6-4650-827d-9b0f2f1e344d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.124676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff201-c9e6-4650-827d-9b0f2f1e344d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.145594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-98eff201-c9e6-4650-827d-9b0f2f1e344d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:58.147000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-98eff201-c9e6-4650-827d-9b0f2f1e344d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:58.147330 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb79301-4e3d-4c1a-b414-0f4113634722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.148021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb79301-4e3d-4c1a-b414-0f4113634722 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.148301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb79301-4e3d-4c1a-b414-0f4113634722 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.148571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb79301-4e3d-4c1a-b414-0f4113634722 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.149327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb79301-4e3d-4c1a-b414-0f4113634722 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:58.149628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb79301-4e3d-4c1a-b414-0f4113634722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:58.151054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb79301-4e3d-4c1a-b414-0f4113634722 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:58.160821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 275/546] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:03:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:58.170816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb79301-4e3d-4c1a-b414-0f4113634722 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.257766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 272/547] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:03:58.283080 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eff201-c9e6-4650-827d-9b0f2f1e344d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.324406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:58.384813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:58.412590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb79301-4e3d-4c1a-b414-0f4113634722 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.427129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eff201-c9e6-4650-827d-9b0f2f1e344d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.454547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f742a6-03d0-4250-a542-489100923742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.455350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f742a6-03d0-4250-a542-489100923742 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.455633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f742a6-03d0-4250-a542-489100923742 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.455906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f742a6-03d0-4250-a542-489100923742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.474465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60884514-255d-4305-b8b1-45df2b4501b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.475723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60884514-255d-4305-b8b1-45df2b4501b1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.476200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60884514-255d-4305-b8b1-45df2b4501b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.476720 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60884514-255d-4305-b8b1-45df2b4501b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.477985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f742a6-03d0-4250-a542-489100923742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:58.479546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f742a6-03d0-4250-a542-489100923742 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:58.490845 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60884514-255d-4305-b8b1-45df2b4501b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-60884514-255d-4305-b8b1-45df2b4501b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:03:58.501716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:58.502667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:58.503441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:58.503441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:58.503441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:58.503441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:58.503441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:03:58.503441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:03:58.503441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:03:58.503441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:03:58.503441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:03:58.503441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:03:58.503441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:03:58.503441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:58.506261 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 273/548] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:03:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-07f742a6-03d0-4250-a542-489100923742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:58.512909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:58.514444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:58.515441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:58.515441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:58.515441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:58.515441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:58.516101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60884514-255d-4305-b8b1-45df2b4501b1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.525818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 276/549] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:58.531889 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f742a6-03d0-4250-a542-489100923742 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.628915 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:58.645811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60884514-255d-4305-b8b1-45df2b4501b1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.705358 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49842c5-9026-450d-a32d-96461f2cb377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.706522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49842c5-9026-450d-a32d-96461f2cb377 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.706522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49842c5-9026-450d-a32d-96461f2cb377 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.706705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49842c5-9026-450d-a32d-96461f2cb377 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.707408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49842c5-9026-450d-a32d-96461f2cb377 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:58.707717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49842c5-9026-450d-a32d-96461f2cb377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:58.709095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49842c5-9026-450d-a32d-96461f2cb377 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:58.729691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 274/550] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:03:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:58.735497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49842c5-9026-450d-a32d-96461f2cb377 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.754208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f742a6-03d0-4250-a542-489100923742 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.806838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:58.822548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58524ec9-b55d-4ffa-aadb-5415e92787c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.823231 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58524ec9-b55d-4ffa-aadb-5415e92787c9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.823517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58524ec9-b55d-4ffa-aadb-5415e92787c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.823794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58524ec9-b55d-4ffa-aadb-5415e92787c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.824881 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58524ec9-b55d-4ffa-aadb-5415e92787c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:58.825495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58524ec9-b55d-4ffa-aadb-5415e92787c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:58.826569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58524ec9-b55d-4ffa-aadb-5415e92787c9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:58.852815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 277/551] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:58.863175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58524ec9-b55d-4ffa-aadb-5415e92787c9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.865385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49842c5-9026-450d-a32d-96461f2cb377 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.901659 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650a2ce-9b3d-43f5-aac8-7076867c25b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.902630 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650a2ce-9b3d-43f5-aac8-7076867c25b7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.903047 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650a2ce-9b3d-43f5-aac8-7076867c25b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.903609 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650a2ce-9b3d-43f5-aac8-7076867c25b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.920696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650a2ce-9b3d-43f5-aac8-7076867c25b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:58.922264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650a2ce-9b3d-43f5-aac8-7076867c25b7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-c650a2ce-9b3d-43f5-aac8-7076867c25b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:03:58.947765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:58.948787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:58.950724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:03:58.950724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:03:58.950724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:03:58.950724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:58.951815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 275/552] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:58.960631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650a2ce-9b3d-43f5-aac8-7076867c25b7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.980122 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:59.083231 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58524ec9-b55d-4ffa-aadb-5415e92787c9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:59.131371 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c715de-9d0d-46f9-8a08-cfaff56917a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:59.132605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c715de-9d0d-46f9-8a08-cfaff56917a3 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:59.133115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c715de-9d0d-46f9-8a08-cfaff56917a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:59.133598 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c715de-9d0d-46f9-8a08-cfaff56917a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:59.151397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c715de-9d0d-46f9-8a08-cfaff56917a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:59.154147 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c715de-9d0d-46f9-8a08-cfaff56917a3 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-b7c715de-9d0d-46f9-8a08-cfaff56917a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:03:59.183429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:59.185377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:59.186451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:59.186451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:59.186451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:03:59.186451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:03:59.186451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:03:59.186451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:03:59.186451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:03:59.186451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:03:59.186451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 18:03:59.186451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:03:59.196187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 278/553] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:03:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:03:59.198369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650a2ce-9b3d-43f5-aac8-7076867c25b7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:59.201685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c715de-9d0d-46f9-8a08-cfaff56917a3 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:59.276924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63a4b5-e25b-4310-b440-12831eb13cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:59.278953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63a4b5-e25b-4310-b440-12831eb13cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:59.281182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63a4b5-e25b-4310-b440-12831eb13cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:59.284443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63a4b5-e25b-4310-b440-12831eb13cd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:59.285652 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63a4b5-e25b-4310-b440-12831eb13cd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:59.286200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63a4b5-e25b-4310-b440-12831eb13cd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:59.287896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63a4b5-e25b-4310-b440-12831eb13cd8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:59.317966 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 276/554] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:59.323226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63a4b5-e25b-4310-b440-12831eb13cd8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:59.323706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:59.417366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c715de-9d0d-46f9-8a08-cfaff56917a3 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:59.450962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:59.471755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249deb50-93b1-4b95-963a-aee4b7553178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:59.472420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249deb50-93b1-4b95-963a-aee4b7553178 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:59.472695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249deb50-93b1-4b95-963a-aee4b7553178 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:59.472968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249deb50-93b1-4b95-963a-aee4b7553178 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:59.473822 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249deb50-93b1-4b95-963a-aee4b7553178 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:59.474128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249deb50-93b1-4b95-963a-aee4b7553178 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:59.475671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249deb50-93b1-4b95-963a-aee4b7553178 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:59.495383 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 279/555] 10.222.0.46 () {56 vars in 1135 bytes} [Thu May 26 18:03:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:59.495590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63a4b5-e25b-4310-b440-12831eb13cd8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:59.498029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8c4cd-e1ef-44f2-b146-b49d1626778e None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:59.498314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8c4cd-e1ef-44f2-b146-b49d1626778e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:59.498587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8c4cd-e1ef-44f2-b146-b49d1626778e None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:59.500142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 280/556] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:03:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:59.505903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8c4cd-e1ef-44f2-b146-b49d1626778e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:59.542964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a8712-b85e-4037-89f8-d79afee0f78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:59.543823 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a8712-b85e-4037-89f8-d79afee0f78b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:59.544143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a8712-b85e-4037-89f8-d79afee0f78b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:59.544472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a8712-b85e-4037-89f8-d79afee0f78b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:59.552378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a8712-b85e-4037-89f8-d79afee0f78b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:59.554000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a8712-b85e-4037-89f8-d79afee0f78b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-293a8712-b85e-4037-89f8-d79afee0f78b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:03:59.570962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:03:59.571963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:03:59.573344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:03:59.573344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:03:59.573344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:03:59.573344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:03:59.573344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:03:59.573344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:03:59.573344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:03:59.573344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:03:59.573344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 18:03:59.573344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:03:59.577466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 277/557] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:03:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:03:59.578930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407b5d-ca66-4463-8982-a913a8caf64c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:59.579252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407b5d-ca66-4463-8982-a913a8caf64c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:59.579529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407b5d-ca66-4463-8982-a913a8caf64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:59.602578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-24407b5d-ca66-4463-8982-a913a8caf64c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:59.603773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-24407b5d-ca66-4463-8982-a913a8caf64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:59.623086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8c4cd-e1ef-44f2-b146-b49d1626778e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:59.634810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:59.665179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eb616-179f-4a56-ba76-6adc5a2d5809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:59.665893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eb616-179f-4a56-ba76-6adc5a2d5809 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:59.666170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eb616-179f-4a56-ba76-6adc5a2d5809 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:59.666449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eb616-179f-4a56-ba76-6adc5a2d5809 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:59.694107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 278/558] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:03:59.695073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407b5d-ca66-4463-8982-a913a8caf64c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:59.747832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65eb616-179f-4a56-ba76-6adc5a2d5809 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:59.749325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65eb616-179f-4a56-ba76-6adc5a2d5809 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:59.773100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:59.823092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407b5d-ca66-4463-8982-a913a8caf64c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:59.860294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cfdca-dcf7-41f4-a3a1-20842046baef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:59.860971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cfdca-dcf7-41f4-a3a1-20842046baef None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:59.861252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cfdca-dcf7-41f4-a3a1-20842046baef None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:59.861528 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cfdca-dcf7-41f4-a3a1-20842046baef None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:59.862442 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 281/559] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:03:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 18:03:59.862978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cfdca-dcf7-41f4-a3a1-20842046baef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:59.862978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cfdca-dcf7-41f4-a3a1-20842046baef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:59.863268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eb616-179f-4a56-ba76-6adc5a2d5809 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:59.864262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cfdca-dcf7-41f4-a3a1-20842046baef None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:59.876772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:59.888266 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 279/560] 10.222.0.46 () {56 vars in 1137 bytes} [Thu May 26 18:03:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:59.889268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cfdca-dcf7-41f4-a3a1-20842046baef None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:59.988042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eb616-179f-4a56-ba76-6adc5a2d5809 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:00.003034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cfdca-dcf7-41f4-a3a1-20842046baef None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:00.018526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:00.030008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aae08-55e1-4120-9335-c3420603cb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:00.030824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aae08-55e1-4120-9335-c3420603cb67 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.031144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aae08-55e1-4120-9335-c3420603cb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.031413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aae08-55e1-4120-9335-c3420603cb67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.045075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883f811-ca0d-4577-a98d-0a6bcd7ad563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:00.046211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883f811-ca0d-4577-a98d-0a6bcd7ad563 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.046613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894aae08-55e1-4120-9335-c3420603cb67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:00.046791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883f811-ca0d-4577-a98d-0a6bcd7ad563 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.047225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883f811-ca0d-4577-a98d-0a6bcd7ad563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-894aae08-55e1-4120-9335-c3420603cb67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:00.055553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:00.056709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:00.057636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:00.057636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:00.057636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:00.057636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:00.057636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:00.057636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:00.057636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:00.057636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:00.057636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:00.057636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:00.057636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:00.057636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:00.061813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 282/561] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:03:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:00.063776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aae08-55e1-4120-9335-c3420603cb67 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:00.066190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883f811-ca0d-4577-a98d-0a6bcd7ad563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:00.067728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883f811-ca0d-4577-a98d-0a6bcd7ad563 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-7883f811-ca0d-4577-a98d-0a6bcd7ad563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:00.094552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:00.095455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:00.096369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:00.096369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:00.096369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:00.096369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:00.096369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 280/562] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:00.097875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883f811-ca0d-4577-a98d-0a6bcd7ad563 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:00.170386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:00.208218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aae08-55e1-4120-9335-c3420603cb67 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:00.214133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883f811-ca0d-4577-a98d-0a6bcd7ad563 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:00.246832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab6383-b103-42e6-ad50-46ca5ec722a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:00.248155 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab6383-b103-42e6-ad50-46ca5ec722a5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.248434 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab6383-b103-42e6-ad50-46ca5ec722a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.248704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab6383-b103-42e6-ad50-46ca5ec722a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.252806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150c7b3-48d8-4c1c-bf76-24081bc558b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:00.253471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150c7b3-48d8-4c1c-bf76-24081bc558b6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.253773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150c7b3-48d8-4c1c-bf76-24081bc558b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.254051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150c7b3-48d8-4c1c-bf76-24081bc558b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.254786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150c7b3-48d8-4c1c-bf76-24081bc558b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:00.255090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150c7b3-48d8-4c1c-bf76-24081bc558b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:00.256450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150c7b3-48d8-4c1c-bf76-24081bc558b6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:00.260115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab6383-b103-42e6-ad50-46ca5ec722a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:00.261594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab6383-b103-42e6-ad50-46ca5ec722a5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:00.271774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 281/563] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:04:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:00.271774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150c7b3-48d8-4c1c-bf76-24081bc558b6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-e9ab6383-b103-42e6-ad50-46ca5ec722a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:00.281483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:00.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:00.283348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:00.283348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:00.283348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:00.283348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:00.283348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 283/564] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:00.284270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d916-d6d2-4817-abb8-644cd06f63d9 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.284549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d916-d6d2-4817-abb8-644cd06f63d9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.284817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d916-d6d2-4817-abb8-644cd06f63d9 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.290237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 284/565] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:00.290237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82d916-d6d2-4817-abb8-644cd06f63d9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:00.319896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:00.395023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150c7b3-48d8-4c1c-bf76-24081bc558b6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:00.430197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a84a2-2360-4bcd-95bf-11285b657b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:00.430844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a84a2-2360-4bcd-95bf-11285b657b16 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.431121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a84a2-2360-4bcd-95bf-11285b657b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.431391 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a84a2-2360-4bcd-95bf-11285b657b16 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.432101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a84a2-2360-4bcd-95bf-11285b657b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:00.432400 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a84a2-2360-4bcd-95bf-11285b657b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:00.434633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a84a2-2360-4bcd-95bf-11285b657b16 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:00.446527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 282/566] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:00.447254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a84a2-2360-4bcd-95bf-11285b657b16 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:00.456314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82d916-d6d2-4817-abb8-644cd06f63d9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:00.480235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:00.492951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13be90f-6068-43d8-baef-c1c41825e0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:00.493814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13be90f-6068-43d8-baef-c1c41825e0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.494092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13be90f-6068-43d8-baef-c1c41825e0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.494363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13be90f-6068-43d8-baef-c1c41825e0a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.506383 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13be90f-6068-43d8-baef-c1c41825e0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:00.507894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13be90f-6068-43d8-baef-c1c41825e0a4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f13be90f-6068-43d8-baef-c1c41825e0a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:00.525418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:00.526400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:00.527290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:00.527290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:00.527290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:00.527290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:00.527290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 285/567] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:00.528610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a602a-f1df-4558-9e5a-704d36d6334f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.528897 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a602a-f1df-4558-9e5a-704d36d6334f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.529286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a602a-f1df-4558-9e5a-704d36d6334f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.539848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:00.546500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b59a602a-f1df-4558-9e5a-704d36d6334f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:00.547793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b59a602a-f1df-4558-9e5a-704d36d6334f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:00.556004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a84a2-2360-4bcd-95bf-11285b657b16 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:00.593405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a435f4c-48d9-4482-9172-551067bf8902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:00.594104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a435f4c-48d9-4482-9172-551067bf8902 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.594387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a435f4c-48d9-4482-9172-551067bf8902 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.594662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a435f4c-48d9-4482-9172-551067bf8902 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.595376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a435f4c-48d9-4482-9172-551067bf8902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:00.595722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a435f4c-48d9-4482-9172-551067bf8902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:00.597055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a435f4c-48d9-4482-9172-551067bf8902 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:00.609050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 283/568] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:00.610344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5efd25-8e90-4d0b-9389-dff6f9f932a5 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.610633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5efd25-8e90-4d0b-9389-dff6f9f932a5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.610986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5efd25-8e90-4d0b-9389-dff6f9f932a5 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.612258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 284/569] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:00.614488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5efd25-8e90-4d0b-9389-dff6f9f932a5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:00.624555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 286/570] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:00.625396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a602a-f1df-4558-9e5a-704d36d6334f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:00.729677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a602a-f1df-4558-9e5a-704d36d6334f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:00.732344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:00.753403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5efd25-8e90-4d0b-9389-dff6f9f932a5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:00.768913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31622eb5-98b0-4d40-88f9-1fed1c17c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:00.769795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31622eb5-98b0-4d40-88f9-1fed1c17c4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.770180 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31622eb5-98b0-4d40-88f9-1fed1c17c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.770572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31622eb5-98b0-4d40-88f9-1fed1c17c4d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.771663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31622eb5-98b0-4d40-88f9-1fed1c17c4d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:00.772130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31622eb5-98b0-4d40-88f9-1fed1c17c4d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:00.774357 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31622eb5-98b0-4d40-88f9-1fed1c17c4d4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:00.793279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 287/571] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:00.793279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31622eb5-98b0-4d40-88f9-1fed1c17c4d4 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:00.794218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9e211-3006-47de-a3f3-9eba54f072de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:00.794218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9e211-3006-47de-a3f3-9eba54f072de None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.794218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9e211-3006-47de-a3f3-9eba54f072de None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.794218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9e211-3006-47de-a3f3-9eba54f072de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.799605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9e211-3006-47de-a3f3-9eba54f072de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:00.801147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9e211-3006-47de-a3f3-9eba54f072de None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-50c9e211-3006-47de-a3f3-9eba54f072de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:00.816399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:00.817545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:00.818518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:00.818518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:00.818518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:00.818518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:00.818518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:00.818518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:00.818518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:00.818518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:00.818518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 285/572] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:00.819094 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7f1d1-14cf-4d90-8c3e-dfda74dc3c7f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.819161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7f1d1-14cf-4d90-8c3e-dfda74dc3c7f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.819454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7f1d1-14cf-4d90-8c3e-dfda74dc3c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.830997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:00.838115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e9a7f1d1-14cf-4d90-8c3e-dfda74dc3c7f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:00.839299 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e9a7f1d1-14cf-4d90-8c3e-dfda74dc3c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:00.899864 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31622eb5-98b0-4d40-88f9-1fed1c17c4d4 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:00.909541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 286/573] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:00.909541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7f1d1-14cf-4d90-8c3e-dfda74dc3c7f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:00.939546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54c937-af47-4ba5-947c-75f74fc9fd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:00.940389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54c937-af47-4ba5-947c-75f74fc9fd12 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:00.940665 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54c937-af47-4ba5-947c-75f74fc9fd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:00.940937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54c937-af47-4ba5-947c-75f74fc9fd12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:00.948057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54c937-af47-4ba5-947c-75f74fc9fd12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:00.950171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54c937-af47-4ba5-947c-75f74fc9fd12 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-cf54c937-af47-4ba5-947c-75f74fc9fd12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:00.965002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:00.973064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:00.974428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:00.974428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:00.974428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:00.974428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:00.974428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:00.974428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:00.974428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:00.974428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:00.974428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 288/574] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:00.974428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54c937-af47-4ba5-947c-75f74fc9fd12 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:01.009659 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:01.025155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7f1d1-14cf-4d90-8c3e-dfda74dc3c7f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:01.065894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd1f7a-f939-4940-b9f2-8529dd604d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:01.066704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd1f7a-f939-4940-b9f2-8529dd604d2c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:01.066981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd1f7a-f939-4940-b9f2-8529dd604d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:01.067271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd1f7a-f939-4940-b9f2-8529dd604d2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:01.070680 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:01.080859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd1f7a-f939-4940-b9f2-8529dd604d2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-55dd1f7a-f939-4940-b9f2-8529dd604d2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:01.088666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:01.090613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 287/575] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:01.092184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd1f7a-f939-4940-b9f2-8529dd604d2c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:01.106823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54c937-af47-4ba5-947c-75f74fc9fd12 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:01.143612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c48ea8-8e87-40a4-9d95-833197e232ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:01.144402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c48ea8-8e87-40a4-9d95-833197e232ab None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:01.144684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c48ea8-8e87-40a4-9d95-833197e232ab None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:01.144955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c48ea8-8e87-40a4-9d95-833197e232ab None admin] PATH_INFO: `/v3/users/glance` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:01.151998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c48ea8-8e87-40a4-9d95-833197e232ab None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:01.153502 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c48ea8-8e87-40a4-9d95-833197e232ab None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c9c48ea8-8e87-40a4-9d95-833197e232ab None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:01.167981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:01.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:01.170614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:01.170614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:01.170614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:01.170614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:01.170614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:01.170614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:01.170614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:01.170614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:01.170614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 18:04:01.170614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:01.173871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 289/576] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:04:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:01.174883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c48ea8-8e87-40a4-9d95-833197e232ab None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:01.196740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd1f7a-f939-4940-b9f2-8529dd604d2c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:01.232739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59edaa7-0d8c-4541-93f6-e274909cc2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:01.233417 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59edaa7-0d8c-4541-93f6-e274909cc2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:01.233690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59edaa7-0d8c-4541-93f6-e274909cc2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:01.233988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59edaa7-0d8c-4541-93f6-e274909cc2e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:01.234769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59edaa7-0d8c-4541-93f6-e274909cc2e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:01.235067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59edaa7-0d8c-4541-93f6-e274909cc2e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:01.236476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59edaa7-0d8c-4541-93f6-e274909cc2e5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:01.249637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 288/577] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:01.250364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59edaa7-0d8c-4541-93f6-e274909cc2e5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:01.280025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:01.298776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c48ea8-8e87-40a4-9d95-833197e232ab None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:01.314916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:01.338134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95433fd6-de9d-4a5f-985c-387cc52fbade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:01.338927 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95433fd6-de9d-4a5f-985c-387cc52fbade None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:01.339315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95433fd6-de9d-4a5f-985c-387cc52fbade None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:01.339593 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95433fd6-de9d-4a5f-985c-387cc52fbade None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:01.350045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95433fd6-de9d-4a5f-985c-387cc52fbade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-95433fd6-de9d-4a5f-985c-387cc52fbade None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:01.358573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:01.359746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:01.360753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:01.360753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:01.360753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:01.360753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:01.360753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:01.360753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:01.360753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:01.360753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:01.360753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:01.360753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:01.360753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:01.360753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:01.361499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59edaa7-0d8c-4541-93f6-e274909cc2e5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:01.364627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 290/578] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:01.365339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95433fd6-de9d-4a5f-985c-387cc52fbade None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:01.395865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba54086-c300-4b58-acae-9cef52ce42dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:01.396527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba54086-c300-4b58-acae-9cef52ce42dc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:01.396809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba54086-c300-4b58-acae-9cef52ce42dc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:01.397085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba54086-c300-4b58-acae-9cef52ce42dc None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:01.397930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba54086-c300-4b58-acae-9cef52ce42dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:01.398219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba54086-c300-4b58-acae-9cef52ce42dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:01.399886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba54086-c300-4b58-acae-9cef52ce42dc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:01.412793 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 289/579] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:01.413638 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba54086-c300-4b58-acae-9cef52ce42dc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:01.470180 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95433fd6-de9d-4a5f-985c-387cc52fbade None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:01.504237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449c6d9-579a-406e-9e40-404cacc902c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:01.504895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449c6d9-579a-406e-9e40-404cacc902c5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:01.505171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449c6d9-579a-406e-9e40-404cacc902c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:01.505450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449c6d9-579a-406e-9e40-404cacc902c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:01.506214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449c6d9-579a-406e-9e40-404cacc902c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:01.506517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449c6d9-579a-406e-9e40-404cacc902c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:01.507894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449c6d9-579a-406e-9e40-404cacc902c5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:01.513357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba54086-c300-4b58-acae-9cef52ce42dc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:01.515800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 291/580] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:04:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:01.516597 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449c6d9-579a-406e-9e40-404cacc902c5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:01.526019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:01.530561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:01.549991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80e9a5-ab72-4618-8078-489778eeedf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:01.550766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80e9a5-ab72-4618-8078-489778eeedf3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:01.551045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80e9a5-ab72-4618-8078-489778eeedf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:01.551315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80e9a5-ab72-4618-8078-489778eeedf3 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:01.552127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80e9a5-ab72-4618-8078-489778eeedf3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:01.552431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80e9a5-ab72-4618-8078-489778eeedf3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:01.553868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80e9a5-ab72-4618-8078-489778eeedf3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:01.571234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 290/581] 10.222.0.46 () {56 vars in 1137 bytes} [Thu May 26 18:04:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:01.571859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80e9a5-ab72-4618-8078-489778eeedf3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:01.623306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449c6d9-579a-406e-9e40-404cacc902c5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:01.657081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1126c2-4472-4dbd-999d-fc4066ac6110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:01.657758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1126c2-4472-4dbd-999d-fc4066ac6110 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:01.658038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1126c2-4472-4dbd-999d-fc4066ac6110 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:01.658308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1126c2-4472-4dbd-999d-fc4066ac6110 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:01.659124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1126c2-4472-4dbd-999d-fc4066ac6110 None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=d656b42c2acb498caa92c77a22fd7efe) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:01.659436 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1126c2-4472-4dbd-999d-fc4066ac6110 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:01.661151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1126c2-4472-4dbd-999d-fc4066ac6110 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:01.677762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 292/582] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:01 2022] GET /identity/v3/role_assignments?user.id=d656b42c2acb498caa92c77a22fd7efe&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:01.677762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1126c2-4472-4dbd-999d-fc4066ac6110 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:01.680452 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80e9a5-ab72-4618-8078-489778eeedf3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:01.728339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69fbfb-0f07-42f8-b51c-a043f4a09b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:01.729027 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69fbfb-0f07-42f8-b51c-a043f4a09b02 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:01.729311 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69fbfb-0f07-42f8-b51c-a043f4a09b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:01.729584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69fbfb-0f07-42f8-b51c-a043f4a09b02 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:01.730392 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69fbfb-0f07-42f8-b51c-a043f4a09b02 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:01.730739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69fbfb-0f07-42f8-b51c-a043f4a09b02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:01.732189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69fbfb-0f07-42f8-b51c-a043f4a09b02 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:01.741354 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 291/583] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:04:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:01.742283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69fbfb-0f07-42f8-b51c-a043f4a09b02 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:01.777259 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:01.786124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:01.817632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1126c2-4472-4dbd-999d-fc4066ac6110 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:01.869165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7f077-a393-44fc-86e5-e1270365d50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:01.870645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7f077-a393-44fc-86e5-e1270365d50e None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:01.871108 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7f077-a393-44fc-86e5-e1270365d50e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:01.872057 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69fbfb-0f07-42f8-b51c-a043f4a09b02 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:01.873281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7f077-a393-44fc-86e5-e1270365d50e None admin] PATH_INFO: `/v3/projects/4be4045293754773bb52bbb82546a1db/users/8c79367697f7451fb804b8641444f6aa/roles/d9bae602bae74c71b6c858434a426a3d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:01.899351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7f077-a393-44fc-86e5-e1270365d50e None admin] RBAC: Authorizing `identity:create_grant(project_id=4be4045293754773bb52bbb82546a1db, user_id=8c79367697f7451fb804b8641444f6aa, role_id=d9bae602bae74c71b6c858434a426a3d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:01.900841 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7f077-a393-44fc-86e5-e1270365d50e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:01.914313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d0d95-7be3-40e4-8420-8d084397dc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:01.915159 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d0d95-7be3-40e4-8420-8d084397dc32 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:01.915425 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d0d95-7be3-40e4-8420-8d084397dc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:01.915718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d0d95-7be3-40e4-8420-8d084397dc32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:01.929768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 293/584] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:04:01 2022] PUT /identity/v3/projects/4be4045293754773bb52bbb82546a1db/users/8c79367697f7451fb804b8641444f6aa/roles/d9bae602bae74c71b6c858434a426a3d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:01.929768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7f077-a393-44fc-86e5-e1270365d50e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:01.940759 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d0d95-7be3-40e4-8420-8d084397dc32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:01.943653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d0d95-7be3-40e4-8420-8d084397dc32 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-288d0d95-7be3-40e4-8420-8d084397dc32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:01.967328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:01.968472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:01.969566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:01.969566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:01.969566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:01.969566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:01.975489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 292/585] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:01.976643 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d0d95-7be3-40e4-8420-8d084397dc32 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:02.135255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7f077-a393-44fc-86e5-e1270365d50e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:02.145922 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:02.163689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d0d95-7be3-40e4-8420-8d084397dc32 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:02.166657 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:02.183632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000e338-5d44-46fa-a0c4-0337caf85bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:02.185143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000e338-5d44-46fa-a0c4-0337caf85bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:02.185494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000e338-5d44-46fa-a0c4-0337caf85bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:02.186277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000e338-5d44-46fa-a0c4-0337caf85bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:02.203791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000e338-5d44-46fa-a0c4-0337caf85bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:02.204317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b56336-1916-49a2-9e48-910226287ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:02.205120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56336-1916-49a2-9e48-910226287ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:02.205402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56336-1916-49a2-9e48-910226287ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:02.205589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000e338-5d44-46fa-a0c4-0337caf85bc2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:02.205745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56336-1916-49a2-9e48-910226287ae5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:02.218016 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b56336-1916-49a2-9e48-910226287ae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:02.219468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b56336-1916-49a2-9e48-910226287ae5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9000e338-5d44-46fa-a0c4-0337caf85bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:02.229528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:02.230783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:02.231890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:02.231890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:02.231890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:02.231890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:02.232198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 294/586] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:02.233407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000e338-5d44-46fa-a0c4-0337caf85bc2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f1b56336-1916-49a2-9e48-910226287ae5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:02.239559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:02.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:02.244869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:02.244869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:02.244869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:02.244869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:02.244869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 293/587] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:02.244869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b56336-1916-49a2-9e48-910226287ae5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:02.363952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000e338-5d44-46fa-a0c4-0337caf85bc2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:02.369151 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b56336-1916-49a2-9e48-910226287ae5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:02.400060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fd835-a7fc-4aed-bea5-b3ffef1601f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:02.400763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fd835-a7fc-4aed-bea5-b3ffef1601f6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:02.401113 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fd835-a7fc-4aed-bea5-b3ffef1601f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:02.401456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fd835-a7fc-4aed-bea5-b3ffef1601f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:02.402353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fd835-a7fc-4aed-bea5-b3ffef1601f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:02.402772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fd835-a7fc-4aed-bea5-b3ffef1601f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:02.404528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fd835-a7fc-4aed-bea5-b3ffef1601f6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:02.411673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214a2de-a31e-4076-a1d2-daa3aa9fab9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:02.412719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214a2de-a31e-4076-a1d2-daa3aa9fab9d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:02.413184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214a2de-a31e-4076-a1d2-daa3aa9fab9d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:02.413660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214a2de-a31e-4076-a1d2-daa3aa9fab9d None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:02.414822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214a2de-a31e-4076-a1d2-daa3aa9fab9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:02.415647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214a2de-a31e-4076-a1d2-daa3aa9fab9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:02.417956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 295/588] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:02.418104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214a2de-a31e-4076-a1d2-daa3aa9fab9d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:02.419242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fd835-a7fc-4aed-bea5-b3ffef1601f6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:02.428369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:02.435100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 294/589] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:02.436363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214a2de-a31e-4076-a1d2-daa3aa9fab9d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:02.454208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:02.526111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fd835-a7fc-4aed-bea5-b3ffef1601f6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:02.563153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a5f5b-d0dc-432b-b57f-da63aa1b86c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:02.563935 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a5f5b-d0dc-432b-b57f-da63aa1b86c4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:02.564279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a5f5b-d0dc-432b-b57f-da63aa1b86c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:02.564642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a5f5b-d0dc-432b-b57f-da63aa1b86c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:02.565436 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a5f5b-d0dc-432b-b57f-da63aa1b86c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:02.565644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214a2de-a31e-4076-a1d2-daa3aa9fab9d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:02.565904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a5f5b-d0dc-432b-b57f-da63aa1b86c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:02.567366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a5f5b-d0dc-432b-b57f-da63aa1b86c4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:02.581345 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 296/590] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:02.582414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a5f5b-d0dc-432b-b57f-da63aa1b86c4 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:02.605787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a13829-91ac-4795-8c3d-7c47768cf296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:02.606697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a13829-91ac-4795-8c3d-7c47768cf296 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:02.607151 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a13829-91ac-4795-8c3d-7c47768cf296 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:02.607527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a13829-91ac-4795-8c3d-7c47768cf296 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:02.615767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a13829-91ac-4795-8c3d-7c47768cf296 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:02.617368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a13829-91ac-4795-8c3d-7c47768cf296 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-51a13829-91ac-4795-8c3d-7c47768cf296 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:02.633667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:02.634893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:02.635961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:02.635961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:02.635961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:02.635961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:02.635961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:02.635961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:02.635961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:02.635961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:02.635961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 18:04:02.635961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:02.638901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 295/591] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 18:04:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:02.639714 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a13829-91ac-4795-8c3d-7c47768cf296 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:02.682307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:02.691637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a5f5b-d0dc-432b-b57f-da63aa1b86c4 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:02.710783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:02.728118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d580f4-f550-409f-811e-64f3b2f0a858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:02.728923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d580f4-f550-409f-811e-64f3b2f0a858 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:02.729207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d580f4-f550-409f-811e-64f3b2f0a858 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:02.729474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d580f4-f550-409f-811e-64f3b2f0a858 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:02.737708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d580f4-f550-409f-811e-64f3b2f0a858 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:02.739301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d580f4-f550-409f-811e-64f3b2f0a858 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f1d580f4-f550-409f-811e-64f3b2f0a858 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:02.755026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:02.757187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:02.759329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:02.759329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:02.759329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:02.759329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:02.759329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:02.759329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:02.759329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:02.759329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:02.759329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 297/592] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:02.759329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d580f4-f550-409f-811e-64f3b2f0a858 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:02.769766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a13829-91ac-4795-8c3d-7c47768cf296 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:02.802472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcddbed-980f-4971-8854-7a1db706524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:02.803278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcddbed-980f-4971-8854-7a1db706524a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:02.803561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcddbed-980f-4971-8854-7a1db706524a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:02.803838 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcddbed-980f-4971-8854-7a1db706524a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:02.812184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcddbed-980f-4971-8854-7a1db706524a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:02.814079 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcddbed-980f-4971-8854-7a1db706524a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-3dcddbed-980f-4971-8854-7a1db706524a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:02.830127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:02.832448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:02.833809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:02.833809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:02.833809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:02.833809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:02.833809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:02.833809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:02.833809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:02.833809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:02.833809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 18:04:02.833809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:02.833809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 296/593] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:02.833809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcddbed-980f-4971-8854-7a1db706524a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:02.870015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d580f4-f550-409f-811e-64f3b2f0a858 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:02.925434 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e422be5-580b-4170-99f2-2c88ae26073f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:02.926158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e422be5-580b-4170-99f2-2c88ae26073f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:02.926755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e422be5-580b-4170-99f2-2c88ae26073f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:02.927041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e422be5-580b-4170-99f2-2c88ae26073f None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:02.928211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e422be5-580b-4170-99f2-2c88ae26073f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:02.928639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e422be5-580b-4170-99f2-2c88ae26073f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:02.930800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e422be5-580b-4170-99f2-2c88ae26073f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:02.952806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcddbed-980f-4971-8854-7a1db706524a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:02.957940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 298/594] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 18:04:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:02.958909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e422be5-580b-4170-99f2-2c88ae26073f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:02.978080 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:02.978283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:02.996353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2100e9e-062e-4d48-991e-773b6bdd2493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:02.997253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2100e9e-062e-4d48-991e-773b6bdd2493 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:02.997537 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2100e9e-062e-4d48-991e-773b6bdd2493 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:02.997850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2100e9e-062e-4d48-991e-773b6bdd2493 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:02.998642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2100e9e-062e-4d48-991e-773b6bdd2493 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:02.998977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2100e9e-062e-4d48-991e-773b6bdd2493 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:03.000419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2100e9e-062e-4d48-991e-773b6bdd2493 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:03.013522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 297/595] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:03.014192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2100e9e-062e-4d48-991e-773b6bdd2493 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:03.086823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e422be5-580b-4170-99f2-2c88ae26073f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:03.128760 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb259a2-553b-4716-b1eb-e8e0a7fd832d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:03.129659 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb259a2-553b-4716-b1eb-e8e0a7fd832d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:03.130113 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb259a2-553b-4716-b1eb-e8e0a7fd832d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:03.130395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb259a2-553b-4716-b1eb-e8e0a7fd832d None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:03.131243 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb259a2-553b-4716-b1eb-e8e0a7fd832d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:03.131555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb259a2-553b-4716-b1eb-e8e0a7fd832d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:03.133036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb259a2-553b-4716-b1eb-e8e0a7fd832d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:03.155160 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 299/596] 10.222.0.46 () {56 vars in 1139 bytes} [Thu May 26 18:04:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:03.156135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb259a2-553b-4716-b1eb-e8e0a7fd832d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:03.156582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2100e9e-062e-4d48-991e-773b6bdd2493 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:03.207181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cefda-28d5-426c-864f-3fa77228ae29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:03.208117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cefda-28d5-426c-864f-3fa77228ae29 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:03.208397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cefda-28d5-426c-864f-3fa77228ae29 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:03.208666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cefda-28d5-426c-864f-3fa77228ae29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:03.224250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cefda-28d5-426c-864f-3fa77228ae29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:03.226644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cefda-28d5-426c-864f-3fa77228ae29 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:03.230695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-094cefda-28d5-426c-864f-3fa77228ae29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:03.251192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:03.252216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:03.253112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:03.253112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:03.253112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:03.253112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:03.260024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 298/597] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:03.260024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cefda-28d5-426c-864f-3fa77228ae29 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:03.275319 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb259a2-553b-4716-b1eb-e8e0a7fd832d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:03.310197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:03.320856 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678d8c8-fa98-4b19-bab3-5f42b3652eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:03.322236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678d8c8-fa98-4b19-bab3-5f42b3652eb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:03.323135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678d8c8-fa98-4b19-bab3-5f42b3652eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:03.323924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678d8c8-fa98-4b19-bab3-5f42b3652eb9 None admin] PATH_INFO: `/v3/projects/4be4045293754773bb52bbb82546a1db/users/4a67120485f74f8bb0ff188cd3abd21c/roles/d9bae602bae74c71b6c858434a426a3d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:03.344037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678d8c8-fa98-4b19-bab3-5f42b3652eb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4be4045293754773bb52bbb82546a1db, user_id=4a67120485f74f8bb0ff188cd3abd21c, role_id=d9bae602bae74c71b6c858434a426a3d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:03.346416 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678d8c8-fa98-4b19-bab3-5f42b3652eb9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:03.370146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 300/598] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:04:03 2022] PUT /identity/v3/projects/4be4045293754773bb52bbb82546a1db/users/4a67120485f74f8bb0ff188cd3abd21c/roles/d9bae602bae74c71b6c858434a426a3d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:03.370146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678d8c8-fa98-4b19-bab3-5f42b3652eb9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:03.421257 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cefda-28d5-426c-864f-3fa77228ae29 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:03.483785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bf5bf-c2f4-4103-ab09-ef718f5b9a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:03.484538 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bf5bf-c2f4-4103-ab09-ef718f5b9a2a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:03.484821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bf5bf-c2f4-4103-ab09-ef718f5b9a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:03.485094 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bf5bf-c2f4-4103-ab09-ef718f5b9a2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:03.499598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bf5bf-c2f4-4103-ab09-ef718f5b9a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:03.501401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bf5bf-c2f4-4103-ab09-ef718f5b9a2a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b55bf5bf-c2f4-4103-ab09-ef718f5b9a2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:03.532507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:03.533477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:03.534372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:03.534372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:03.534372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:03.534372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:03.544283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 299/599] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:03.549701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bf5bf-c2f4-4103-ab09-ef718f5b9a2a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:03.599198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678d8c8-fa98-4b19-bab3-5f42b3652eb9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:03.642080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:03.674351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272e3d3-48d3-4a0c-acc5-02afdb0f6bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:03.675039 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272e3d3-48d3-4a0c-acc5-02afdb0f6bea None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:03.675323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272e3d3-48d3-4a0c-acc5-02afdb0f6bea None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:03.675601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272e3d3-48d3-4a0c-acc5-02afdb0f6bea None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:03.677251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272e3d3-48d3-4a0c-acc5-02afdb0f6bea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:03.677613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272e3d3-48d3-4a0c-acc5-02afdb0f6bea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:03.679941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272e3d3-48d3-4a0c-acc5-02afdb0f6bea None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:03.689371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:03.705971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 301/600] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:03.715950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272e3d3-48d3-4a0c-acc5-02afdb0f6bea None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:03.717816 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bf5bf-c2f4-4103-ab09-ef718f5b9a2a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:03.791477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66aecf2-7053-4cc6-bd72-4b477ab52870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:03.792063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66aecf2-7053-4cc6-bd72-4b477ab52870 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:03.792341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66aecf2-7053-4cc6-bd72-4b477ab52870 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:03.792605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66aecf2-7053-4cc6-bd72-4b477ab52870 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:03.793364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66aecf2-7053-4cc6-bd72-4b477ab52870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:03.793785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66aecf2-7053-4cc6-bd72-4b477ab52870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:03.795369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66aecf2-7053-4cc6-bd72-4b477ab52870 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:03.822076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 300/601] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:03.822887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66aecf2-7053-4cc6-bd72-4b477ab52870 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:03.854247 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272e3d3-48d3-4a0c-acc5-02afdb0f6bea None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:03.894203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338378b-553e-459f-a1fb-bef347a57e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:03.895002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338378b-553e-459f-a1fb-bef347a57e46 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:03.895283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338378b-553e-459f-a1fb-bef347a57e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:03.895554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338378b-553e-459f-a1fb-bef347a57e46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:03.911196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338378b-553e-459f-a1fb-bef347a57e46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:03.912690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338378b-553e-459f-a1fb-bef347a57e46 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-2338378b-553e-459f-a1fb-bef347a57e46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:03.941779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:03.943400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:03.944403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:03.944403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:03.944403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:03.944403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:03.944403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:03.944403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:03.944403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:03.944403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:03.948634 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 302/602] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:03.959044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338378b-553e-459f-a1fb-bef347a57e46 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:04.038461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:04.053690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66aecf2-7053-4cc6-bd72-4b477ab52870 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:04.135336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a6d63-1186-4ee4-bb25-3ef9cc3115be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:04.136133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a6d63-1186-4ee4-bb25-3ef9cc3115be None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:04.136410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a6d63-1186-4ee4-bb25-3ef9cc3115be None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:04.136678 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a6d63-1186-4ee4-bb25-3ef9cc3115be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:04.151298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338378b-553e-459f-a1fb-bef347a57e46 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:04.154888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a6d63-1186-4ee4-bb25-3ef9cc3115be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:04.156339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a6d63-1186-4ee4-bb25-3ef9cc3115be None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:04.189975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b043e-2224-4ed5-9347-8e661444081f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:04.190691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b043e-2224-4ed5-9347-8e661444081f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:04.190970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b043e-2224-4ed5-9347-8e661444081f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:04.191239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b043e-2224-4ed5-9347-8e661444081f None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:04.192074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b043e-2224-4ed5-9347-8e661444081f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:04.192382 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b043e-2224-4ed5-9347-8e661444081f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:04.193861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b043e-2224-4ed5-9347-8e661444081f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-804a6d63-1186-4ee4-bb25-3ef9cc3115be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:04.195072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:04.195941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:04.196810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:04.196810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:04.196810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:04.196810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:04.196810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:04.196810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:04.196810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:04.196810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:04.196810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 301/603] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:04.204321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a6d63-1186-4ee4-bb25-3ef9cc3115be None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:04.217277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 303/604] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:04.223521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:04.227374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b043e-2224-4ed5-9347-8e661444081f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:04.425133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a6d63-1186-4ee4-bb25-3ef9cc3115be None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:04.429010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:04.434955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b043e-2224-4ed5-9347-8e661444081f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:04.472176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d248f12-5074-4a9a-be87-aa72c3b8c8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:04.472914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d248f12-5074-4a9a-be87-aa72c3b8c8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:04.473211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d248f12-5074-4a9a-be87-aa72c3b8c8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:04.473481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d248f12-5074-4a9a-be87-aa72c3b8c8e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:04.474163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe396d-9661-4eff-b037-6749dbccfb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:04.474284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d248f12-5074-4a9a-be87-aa72c3b8c8e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:04.474589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d248f12-5074-4a9a-be87-aa72c3b8c8e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:04.475575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe396d-9661-4eff-b037-6749dbccfb7f None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:04.476021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d248f12-5074-4a9a-be87-aa72c3b8c8e5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:04.476279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe396d-9661-4eff-b037-6749dbccfb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:04.476743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe396d-9661-4eff-b037-6749dbccfb7f None admin] PATH_INFO: `/v3/projects/4be4045293754773bb52bbb82546a1db/users/d420e808730d484f80963322c5fc6856/roles/d9bae602bae74c71b6c858434a426a3d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:04.497815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 302/605] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:04.501893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff5478-632b-470c-a396-477594308d33 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:04.502901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff5478-632b-470c-a396-477594308d33 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:04.503284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe396d-9661-4eff-b037-6749dbccfb7f None admin] RBAC: Authorizing `identity:create_grant(project_id=4be4045293754773bb52bbb82546a1db, user_id=d420e808730d484f80963322c5fc6856, role_id=d9bae602bae74c71b6c858434a426a3d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:04.504715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff5478-632b-470c-a396-477594308d33 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:04.507302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe396d-9661-4eff-b037-6749dbccfb7f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:04.509144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 303/606] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:04.510914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff5478-632b-470c-a396-477594308d33 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:04.536694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 304/607] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:04:04 2022] PUT /identity/v3/projects/4be4045293754773bb52bbb82546a1db/users/d420e808730d484f80963322c5fc6856/roles/d9bae602bae74c71b6c858434a426a3d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:04.542061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcbc21-f0c7-4394-af4c-bbdb4b229779 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:04.542061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcbc21-f0c7-4394-af4c-bbdb4b229779 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:04.542061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcbc21-f0c7-4394-af4c-bbdb4b229779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:04.563310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-23bcbc21-f0c7-4394-af4c-bbdb4b229779 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:04.565328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-23bcbc21-f0c7-4394-af4c-bbdb4b229779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:04.599952 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:04.650690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff5478-632b-470c-a396-477594308d33 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:04.677768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 305/608] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:04.677768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbc886-55dd-4b15-9a57-d455173a5cbd None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:04.677768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbc886-55dd-4b15-9a57-d455173a5cbd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:04.677768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbc886-55dd-4b15-9a57-d455173a5cbd None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:04.677768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 306/609] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:04.683857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc9979-bb2c-4673-b739-8d33ea9bc4d8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:04.684201 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc9979-bb2c-4673-b739-8d33ea9bc4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:04.684672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc9979-bb2c-4673-b739-8d33ea9bc4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:04.711672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8118dc-1183-4aea-a052-e035fa1250dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:04.712094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-53cc9979-bb2c-4673-b739-8d33ea9bc4d8 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:04.713330 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-53cc9979-bb2c-4673-b739-8d33ea9bc4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:04.713612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8118dc-1183-4aea-a052-e035fa1250dd None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:04.714236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8118dc-1183-4aea-a052-e035fa1250dd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:04.714749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8118dc-1183-4aea-a052-e035fa1250dd None admin] PATH_INFO: `/v3/projects/4be4045293754773bb52bbb82546a1db/users/f668c4c6745e4f1eae20d8a3a12cf0cc/roles/d9bae602bae74c71b6c858434a426a3d` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:04.735076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8118dc-1183-4aea-a052-e035fa1250dd None admin] RBAC: Authorizing `identity:create_grant(project_id=4be4045293754773bb52bbb82546a1db, user_id=f668c4c6745e4f1eae20d8a3a12cf0cc, role_id=d9bae602bae74c71b6c858434a426a3d)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:04.737055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8118dc-1183-4aea-a052-e035fa1250dd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:04.756614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 304/610] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:04:04 2022] PUT /identity/v3/projects/4be4045293754773bb52bbb82546a1db/users/f668c4c6745e4f1eae20d8a3a12cf0cc/roles/d9bae602bae74c71b6c858434a426a3d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:04.758894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8118dc-1183-4aea-a052-e035fa1250dd None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:04.810430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 307/611] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:04.844702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc9979-bb2c-4673-b739-8d33ea9bc4d8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:04.862239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:04.906500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8118dc-1183-4aea-a052-e035fa1250dd None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:04.926330 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:04.978937 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d554fcb-7035-4720-b41e-ecda9a10c7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:04.979936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d554fcb-7035-4720-b41e-ecda9a10c7ab None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:04.980187 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d554fcb-7035-4720-b41e-ecda9a10c7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:04.980488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d554fcb-7035-4720-b41e-ecda9a10c7ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:04.998596 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d554fcb-7035-4720-b41e-ecda9a10c7ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:05.001007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc9979-bb2c-4673-b739-8d33ea9bc4d8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-5d554fcb-7035-4720-b41e-ecda9a10c7ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:05.008105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:05.009069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:05.009920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 305/612] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:05.018307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d554fcb-7035-4720-b41e-ecda9a10c7ab None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:05.047769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb64696-f877-45ee-8b36-0b3e56f8ca62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:05.048581 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb64696-f877-45ee-8b36-0b3e56f8ca62 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:05.048868 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb64696-f877-45ee-8b36-0b3e56f8ca62 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:05.049139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb64696-f877-45ee-8b36-0b3e56f8ca62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:05.061005 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb64696-f877-45ee-8b36-0b3e56f8ca62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-5cb64696-f877-45ee-8b36-0b3e56f8ca62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:05.068947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:05.069974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:05.070819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 308/613] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:05.076720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb64696-f877-45ee-8b36-0b3e56f8ca62 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:05.152040 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:05.166556 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d554fcb-7035-4720-b41e-ecda9a10c7ab None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:05.190355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb64696-f877-45ee-8b36-0b3e56f8ca62 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:05.213138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db896d1a-623c-4ef3-9701-fa77929c4fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:05.213839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db896d1a-623c-4ef3-9701-fa77929c4fdf None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:05.214123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db896d1a-623c-4ef3-9701-fa77929c4fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:05.214397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db896d1a-623c-4ef3-9701-fa77929c4fdf None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:05.215373 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db896d1a-623c-4ef3-9701-fa77929c4fdf None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:05.216173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db896d1a-623c-4ef3-9701-fa77929c4fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:05.217623 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db896d1a-623c-4ef3-9701-fa77929c4fdf None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:05.227331 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 306/614] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:04:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:05.232001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a745be-d394-4aa5-b0f7-065ea7d6f90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:05.233102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a745be-d394-4aa5-b0f7-065ea7d6f90f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:05.233570 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a745be-d394-4aa5-b0f7-065ea7d6f90f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:05.234062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a745be-d394-4aa5-b0f7-065ea7d6f90f None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:05.235379 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a745be-d394-4aa5-b0f7-065ea7d6f90f None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:05.235933 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a745be-d394-4aa5-b0f7-065ea7d6f90f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:05.238338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a745be-d394-4aa5-b0f7-065ea7d6f90f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:05.239012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db896d1a-623c-4ef3-9701-fa77929c4fdf None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:05.254449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 309/615] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:04:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:05.263111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a745be-d394-4aa5-b0f7-065ea7d6f90f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:05.320988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:05.427735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a745be-d394-4aa5-b0f7-065ea7d6f90f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:05.445717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db896d1a-623c-4ef3-9701-fa77929c4fdf None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:05.485614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:05.502580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b831fd-33a5-4e7b-be79-187bc03a0370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:05.504001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b831fd-33a5-4e7b-be79-187bc03a0370 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:05.504559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b831fd-33a5-4e7b-be79-187bc03a0370 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:05.505178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b831fd-33a5-4e7b-be79-187bc03a0370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:05.505515 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2ccef-dc83-40ec-89ed-9f363e06161d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:05.506414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2ccef-dc83-40ec-89ed-9f363e06161d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:05.506801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2ccef-dc83-40ec-89ed-9f363e06161d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:05.507097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2ccef-dc83-40ec-89ed-9f363e06161d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:05.518877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2ccef-dc83-40ec-89ed-9f363e06161d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:05.521896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2ccef-dc83-40ec-89ed-9f363e06161d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:05.533887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b831fd-33a5-4e7b-be79-187bc03a0370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:05.535919 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b831fd-33a5-4e7b-be79-187bc03a0370 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-4ce2ccef-dc83-40ec-89ed-9f363e06161d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:05.554378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:05.555523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:05.556522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:05.556522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:05.556522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:05.556522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f2b831fd-33a5-4e7b-be79-187bc03a0370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:05.568175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:05.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:05.570312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:05.570312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:05.570312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:05.570312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:05.571537 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 307/616] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:05.573181 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 310/617] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:05.578851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2ccef-dc83-40ec-89ed-9f363e06161d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:05.584270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b831fd-33a5-4e7b-be79-187bc03a0370 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:05.743611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:05.768073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2ccef-dc83-40ec-89ed-9f363e06161d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:05.781262 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b831fd-33a5-4e7b-be79-187bc03a0370 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:05.836296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0b02d-40c1-4128-a42c-3d9d71bb8dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:05.837417 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0b02d-40c1-4128-a42c-3d9d71bb8dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:05.838755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0b02d-40c1-4128-a42c-3d9d71bb8dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:05.839654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0b02d-40c1-4128-a42c-3d9d71bb8dd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:05.841714 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0b02d-40c1-4128-a42c-3d9d71bb8dd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:05.843046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0b02d-40c1-4128-a42c-3d9d71bb8dd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:05.845191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0b02d-40c1-4128-a42c-3d9d71bb8dd2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:05.848099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4056be5-4c20-43d0-8da1-4d2de13f620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:05.848980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4056be5-4c20-43d0-8da1-4d2de13f620b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:05.849411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4056be5-4c20-43d0-8da1-4d2de13f620b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:05.849855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4056be5-4c20-43d0-8da1-4d2de13f620b None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:05.850830 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4056be5-4c20-43d0-8da1-4d2de13f620b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:05.852203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4056be5-4c20-43d0-8da1-4d2de13f620b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:05.853803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4056be5-4c20-43d0-8da1-4d2de13f620b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:05.871733 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 308/618] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:05.878790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 311/619] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:05.880023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0b02d-40c1-4128-a42c-3d9d71bb8dd2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:05.889058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4056be5-4c20-43d0-8da1-4d2de13f620b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:05.973659 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:06.035389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4056be5-4c20-43d0-8da1-4d2de13f620b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:06.073139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e95ed9-401b-4ee2-b946-0119267583e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:06.074521 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e95ed9-401b-4ee2-b946-0119267583e1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:06.075014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e95ed9-401b-4ee2-b946-0119267583e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:06.075570 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e95ed9-401b-4ee2-b946-0119267583e1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:06.083846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e95ed9-401b-4ee2-b946-0119267583e1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:06.085592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e95ed9-401b-4ee2-b946-0119267583e1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-14e95ed9-401b-4ee2-b946-0119267583e1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:06.104483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:06.105624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:06.107125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:06.107125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:06.107125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:06.107125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:06.107125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:06.107125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:06.107125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:06.107125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:06.107125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 18:04:06.107125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:06.109460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 312/620] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:04:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:06.111870 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0b02d-40c1-4128-a42c-3d9d71bb8dd2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:06.114997 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975c3ba-d343-43bd-ad62-486a103e3382 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:06.115284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975c3ba-d343-43bd-ad62-486a103e3382 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:06.115555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975c3ba-d343-43bd-ad62-486a103e3382 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:06.116991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 313/621] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:06.117626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975c3ba-d343-43bd-ad62-486a103e3382 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:06.148826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8bfae-dc57-467b-a440-bd08a2ea1517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:06.149638 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8bfae-dc57-467b-a440-bd08a2ea1517 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:06.149928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8bfae-dc57-467b-a440-bd08a2ea1517 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:06.150198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8bfae-dc57-467b-a440-bd08a2ea1517 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:06.159245 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8bfae-dc57-467b-a440-bd08a2ea1517 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:06.160951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8bfae-dc57-467b-a440-bd08a2ea1517 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-06e8bfae-dc57-467b-a440-bd08a2ea1517 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:06.175640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:06.177033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:06.178005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:06.178005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:06.178005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:06.178005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:06.178005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:06.178005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:06.178005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:06.178005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:06.178005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 18:04:06.178005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:06.178005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 309/622] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:04:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:06.178588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f27a19-d4cf-4ad3-a2f1-5b0428c02b3f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:06.180351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f27a19-d4cf-4ad3-a2f1-5b0428c02b3f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:06.180351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f27a19-d4cf-4ad3-a2f1-5b0428c02b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:06.196818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b5f27a19-d4cf-4ad3-a2f1-5b0428c02b3f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:06.198121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b5f27a19-d4cf-4ad3-a2f1-5b0428c02b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:06.200956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:06.247981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975c3ba-d343-43bd-ad62-486a103e3382 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:06.250216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:06.271707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 310/623] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:06.272712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f27a19-d4cf-4ad3-a2f1-5b0428c02b3f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:06.291321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea17c7b-342f-4a7b-9d6d-3d1debb2bf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:06.292470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea17c7b-342f-4a7b-9d6d-3d1debb2bf72 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:06.293022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea17c7b-342f-4a7b-9d6d-3d1debb2bf72 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:06.293475 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea17c7b-342f-4a7b-9d6d-3d1debb2bf72 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:06.294594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea17c7b-342f-4a7b-9d6d-3d1debb2bf72 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:06.295086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea17c7b-342f-4a7b-9d6d-3d1debb2bf72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:06.296880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea17c7b-342f-4a7b-9d6d-3d1debb2bf72 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:06.322166 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 314/624] 10.222.0.46 () {56 vars in 1135 bytes} [Thu May 26 18:04:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:06.328023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea17c7b-342f-4a7b-9d6d-3d1debb2bf72 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:06.378995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f27a19-d4cf-4ad3-a2f1-5b0428c02b3f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:06.414774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02127e-94b2-4975-977e-dc593fbb9c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:06.415557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02127e-94b2-4975-977e-dc593fbb9c06 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:06.415908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02127e-94b2-4975-977e-dc593fbb9c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:06.416250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02127e-94b2-4975-977e-dc593fbb9c06 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:06.417394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02127e-94b2-4975-977e-dc593fbb9c06 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:06.417507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02127e-94b2-4975-977e-dc593fbb9c06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:06.419603 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02127e-94b2-4975-977e-dc593fbb9c06 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:06.427979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:06.439771 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 311/625] 10.222.0.46 () {56 vars in 1137 bytes} [Thu May 26 18:04:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:06.440627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02127e-94b2-4975-977e-dc593fbb9c06 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:06.449056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea17c7b-342f-4a7b-9d6d-3d1debb2bf72 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:06.487332 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fc21d-e649-4298-ba01-489229e05aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:06.488441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fc21d-e649-4298-ba01-489229e05aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:06.489005 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fc21d-e649-4298-ba01-489229e05aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:06.489517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fc21d-e649-4298-ba01-489229e05aa0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:06.502934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5fc21d-e649-4298-ba01-489229e05aa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-da5fc21d-e649-4298-ba01-489229e05aa0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:06.512998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:06.514042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:06.514888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:06.514888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:06.514888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:06.514888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:06.514888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:06.514888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:06.514888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:06.514888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:06.514888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:06.514888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:06.514888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:06.514888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:06.517436 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 315/626] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:06.518525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fc21d-e649-4298-ba01-489229e05aa0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:06.544236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:06.554037 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02127e-94b2-4975-977e-dc593fbb9c06 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:06.592393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8fad3-f14c-4919-895f-a31bcc3c799f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:06.593559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8fad3-f14c-4919-895f-a31bcc3c799f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:06.594090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8fad3-f14c-4919-895f-a31bcc3c799f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:06.594561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8fad3-f14c-4919-895f-a31bcc3c799f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:06.605662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8fad3-f14c-4919-895f-a31bcc3c799f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:06.607559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8fad3-f14c-4919-895f-a31bcc3c799f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-2ac8fad3-f14c-4919-895f-a31bcc3c799f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:06.627274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:06.628228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:06.629112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:06.629112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:06.629112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:06.629112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:06.631449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 312/627] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:06.632442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8fad3-f14c-4919-895f-a31bcc3c799f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:06.637616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fc21d-e649-4298-ba01-489229e05aa0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:06.677164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bc206-e7ab-4a40-b3c0-94d5c73514c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:06.677877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bc206-e7ab-4a40-b3c0-94d5c73514c1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:06.678160 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bc206-e7ab-4a40-b3c0-94d5c73514c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:06.678431 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bc206-e7ab-4a40-b3c0-94d5c73514c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:06.690747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:06.752791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8fad3-f14c-4919-895f-a31bcc3c799f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:06.764515 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bc206-e7ab-4a40-b3c0-94d5c73514c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:06.765786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bc206-e7ab-4a40-b3c0-94d5c73514c1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:06.789220 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:06.795877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0542da-1c8c-4715-bda7-8bc867283091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:06.796853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0542da-1c8c-4715-bda7-8bc867283091 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:06.797348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0542da-1c8c-4715-bda7-8bc867283091 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:06.797871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0542da-1c8c-4715-bda7-8bc867283091 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:06.799008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0542da-1c8c-4715-bda7-8bc867283091 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:06.799516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0542da-1c8c-4715-bda7-8bc867283091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:06.801121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0542da-1c8c-4715-bda7-8bc867283091 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:06.810894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 313/628] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:04:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:06.812206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0542da-1c8c-4715-bda7-8bc867283091 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:06.880506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 316/629] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:04:06.881465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bc206-e7ab-4a40-b3c0-94d5c73514c1 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:06.923073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0542da-1c8c-4715-bda7-8bc867283091 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:06.934951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:06.961203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc23939-d76c-48f8-a2e4-87b602965f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:06.961582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc23939-d76c-48f8-a2e4-87b602965f35 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:06.962019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc23939-d76c-48f8-a2e4-87b602965f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:06.962390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc23939-d76c-48f8-a2e4-87b602965f35 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:06.963211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc23939-d76c-48f8-a2e4-87b602965f35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:06.963616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc23939-d76c-48f8-a2e4-87b602965f35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:06.965632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc23939-d76c-48f8-a2e4-87b602965f35 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:06.984196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 314/630] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:06.984196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc23939-d76c-48f8-a2e4-87b602965f35 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:07.000631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bc206-e7ab-4a40-b3c0-94d5c73514c1 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:07.023692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:07.040259 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4f907-0d85-41d0-bd46-2f179d159568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:07.041077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4f907-0d85-41d0-bd46-2f179d159568 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.041365 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4f907-0d85-41d0-bd46-2f179d159568 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.041643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4f907-0d85-41d0-bd46-2f179d159568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.053925 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4f907-0d85-41d0-bd46-2f179d159568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:07.055333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4f907-0d85-41d0-bd46-2f179d159568 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-b4b4f907-0d85-41d0-bd46-2f179d159568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:07.075194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:07.076186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:07.077660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:07.077660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:07.077660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:07.077660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:07.077660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 317/631] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:07.077985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4f907-0d85-41d0-bd46-2f179d159568 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:07.094905 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc23939-d76c-48f8-a2e4-87b602965f35 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:07.140659 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b85437-261e-4c50-96b7-843730e480a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:07.142430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b85437-261e-4c50-96b7-843730e480a4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.142934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b85437-261e-4c50-96b7-843730e480a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.143429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b85437-261e-4c50-96b7-843730e480a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.164835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b85437-261e-4c50-96b7-843730e480a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:07.167882 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b85437-261e-4c50-96b7-843730e480a4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:07.193942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4f907-0d85-41d0-bd46-2f179d159568 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-56b85437-261e-4c50-96b7-843730e480a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:07.206231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:07.207468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:07.208366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:07.208366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:07.208366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:07.208366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:07.209058 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 315/632] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:07.210739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b85437-261e-4c50-96b7-843730e480a4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:07.231047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec7292-46f9-442b-8a95-ee62e8bc87a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:07.231855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec7292-46f9-442b-8a95-ee62e8bc87a7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.232133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec7292-46f9-442b-8a95-ee62e8bc87a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.232443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec7292-46f9-442b-8a95-ee62e8bc87a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.240642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec7292-46f9-442b-8a95-ee62e8bc87a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:07.242234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec7292-46f9-442b-8a95-ee62e8bc87a7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:07.251448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a0ec7292-46f9-442b-8a95-ee62e8bc87a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:07.259800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:07.260784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:07.262495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:07.262495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:07.262495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:07.262495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:07.262495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:07.262495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:07.262495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:07.262495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:07.266280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 318/633] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:07.267022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec7292-46f9-442b-8a95-ee62e8bc87a7 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:07.302034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:07.353912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b85437-261e-4c50-96b7-843730e480a4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:07.378907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec7292-46f9-442b-8a95-ee62e8bc87a7 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:07.398747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca936a1-549d-46dc-a3ef-5e222416a22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:07.399428 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca936a1-549d-46dc-a3ef-5e222416a22d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.399712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca936a1-549d-46dc-a3ef-5e222416a22d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.399986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca936a1-549d-46dc-a3ef-5e222416a22d None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.401153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca936a1-549d-46dc-a3ef-5e222416a22d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:07.401465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca936a1-549d-46dc-a3ef-5e222416a22d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:07.403472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca936a1-549d-46dc-a3ef-5e222416a22d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:07.419700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 316/634] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:07.419700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca936a1-549d-46dc-a3ef-5e222416a22d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:07.422206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e5f46-fa36-41e9-9721-b7897df157f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:07.422982 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e5f46-fa36-41e9-9721-b7897df157f7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.423255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e5f46-fa36-41e9-9721-b7897df157f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.424270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e5f46-fa36-41e9-9721-b7897df157f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.424270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e5f46-fa36-41e9-9721-b7897df157f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:07.424563 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e5f46-fa36-41e9-9721-b7897df157f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:07.425962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e5f46-fa36-41e9-9721-b7897df157f7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:07.438906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 319/635] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:07.444264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2455cd4-1987-4fed-b6bd-a3a9fca3170f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.444549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2455cd4-1987-4fed-b6bd-a3a9fca3170f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.444884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2455cd4-1987-4fed-b6bd-a3a9fca3170f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.446083 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 320/636] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:07.446766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2455cd4-1987-4fed-b6bd-a3a9fca3170f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:07.512571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:07.537107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca936a1-549d-46dc-a3ef-5e222416a22d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:07.550957 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:07.561337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2455cd4-1987-4fed-b6bd-a3a9fca3170f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:07.580282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a451e-55b1-43bf-b88f-1217f325ad33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:07.580880 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a451e-55b1-43bf-b88f-1217f325ad33 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.581160 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a451e-55b1-43bf-b88f-1217f325ad33 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.581644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a451e-55b1-43bf-b88f-1217f325ad33 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.582267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a451e-55b1-43bf-b88f-1217f325ad33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:07.582576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a451e-55b1-43bf-b88f-1217f325ad33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:07.584055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a451e-55b1-43bf-b88f-1217f325ad33 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:07.598242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 317/637] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:07.598972 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a451e-55b1-43bf-b88f-1217f325ad33 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:07.608578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058b8d5-25a3-4390-96bc-85ed4f000de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:07.609386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058b8d5-25a3-4390-96bc-85ed4f000de7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.609672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058b8d5-25a3-4390-96bc-85ed4f000de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.609980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058b8d5-25a3-4390-96bc-85ed4f000de7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.617522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058b8d5-25a3-4390-96bc-85ed4f000de7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:07.619042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058b8d5-25a3-4390-96bc-85ed4f000de7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a058b8d5-25a3-4390-96bc-85ed4f000de7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:07.634934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:07.636017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:07.637008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:07.637008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:07.637008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:07.637008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:07.637008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:07.637008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:07.637008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:07.637008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:07.637008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 18:04:07.637008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:07.637008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 321/638] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:04:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:07.637680 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f726a-e927-468a-b299-78be7eaa4e22 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.638283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f726a-e927-468a-b299-78be7eaa4e22 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.638283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f726a-e927-468a-b299-78be7eaa4e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.658840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8d6f726a-e927-468a-b299-78be7eaa4e22 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:07.660071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8d6f726a-e927-468a-b299-78be7eaa4e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:07.713095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a451e-55b1-43bf-b88f-1217f325ad33 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:07.749762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 322/639] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:07.749762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f726a-e927-468a-b299-78be7eaa4e22 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:07.761486 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe881f8-ed35-49e5-be52-60b305741c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:07.762410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe881f8-ed35-49e5-be52-60b305741c42 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.762786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe881f8-ed35-49e5-be52-60b305741c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.763064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe881f8-ed35-49e5-be52-60b305741c42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.770506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe881f8-ed35-49e5-be52-60b305741c42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:07.771940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe881f8-ed35-49e5-be52-60b305741c42 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ffe881f8-ed35-49e5-be52-60b305741c42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:07.786572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:07.787554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:07.794028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:07.794028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:07.794028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:07.794028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:07.794028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:07.794028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:07.794028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:07.794028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:07.794028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 318/640] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:07.794028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe881f8-ed35-49e5-be52-60b305741c42 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:07.794028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:07.828568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:07.847684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f726a-e927-468a-b299-78be7eaa4e22 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:07.881588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48a507-f2b7-417d-a6ec-df8541fc2677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:07.882503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48a507-f2b7-417d-a6ec-df8541fc2677 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.882782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48a507-f2b7-417d-a6ec-df8541fc2677 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.883053 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48a507-f2b7-417d-a6ec-df8541fc2677 None admin] PATH_INFO: `/v3/projects/4be4045293754773bb52bbb82546a1db/users/d656b42c2acb498caa92c77a22fd7efe/roles/d9bae602bae74c71b6c858434a426a3d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.891078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe881f8-ed35-49e5-be52-60b305741c42 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:07.898426 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48a507-f2b7-417d-a6ec-df8541fc2677 None admin] RBAC: Authorizing `identity:create_grant(project_id=4be4045293754773bb52bbb82546a1db, user_id=d656b42c2acb498caa92c77a22fd7efe, role_id=d9bae602bae74c71b6c858434a426a3d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:07.899886 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48a507-f2b7-417d-a6ec-df8541fc2677 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:07.914263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 323/641] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:04:07 2022] PUT /identity/v3/projects/4be4045293754773bb52bbb82546a1db/users/d656b42c2acb498caa92c77a22fd7efe/roles/d9bae602bae74c71b6c858434a426a3d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:07.915567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd2877-7110-4e51-b532-197ee464f5dc None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.915855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd2877-7110-4e51-b532-197ee464f5dc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.916127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd2877-7110-4e51-b532-197ee464f5dc None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.917555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 324/642] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:07.918441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd2877-7110-4e51-b532-197ee464f5dc None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:07.938758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448b8cb-6a3e-4b76-9cb5-20ad72afd5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:07.939448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448b8cb-6a3e-4b76-9cb5-20ad72afd5eb None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:07.939731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448b8cb-6a3e-4b76-9cb5-20ad72afd5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:07.940002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448b8cb-6a3e-4b76-9cb5-20ad72afd5eb None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:07.940854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448b8cb-6a3e-4b76-9cb5-20ad72afd5eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:07.941158 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448b8cb-6a3e-4b76-9cb5-20ad72afd5eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:07.942732 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448b8cb-6a3e-4b76-9cb5-20ad72afd5eb None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:07.964085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 319/643] 10.222.0.46 () {56 vars in 1137 bytes} [Thu May 26 18:04:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:07.965013 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448b8cb-6a3e-4b76-9cb5-20ad72afd5eb None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:08.032496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:08.054324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd2877-7110-4e51-b532-197ee464f5dc None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:08.078550 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448b8cb-6a3e-4b76-9cb5-20ad72afd5eb None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:08.089605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:08.099420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2b05f-e59d-48b6-a9d7-f0b3ae44e7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:08.100946 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2b05f-e59d-48b6-a9d7-f0b3ae44e7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:08.101629 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2b05f-e59d-48b6-a9d7-f0b3ae44e7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:08.103232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2b05f-e59d-48b6-a9d7-f0b3ae44e7b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:08.116403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2b05f-e59d-48b6-a9d7-f0b3ae44e7b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:08.122249 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48149d5-04ab-4af9-a42d-a3dca198be8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:08.122911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48149d5-04ab-4af9-a42d-a3dca198be8d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:08.123194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48149d5-04ab-4af9-a42d-a3dca198be8d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:08.123482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48149d5-04ab-4af9-a42d-a3dca198be8d None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:08.124250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48149d5-04ab-4af9-a42d-a3dca198be8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:08.124560 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48149d5-04ab-4af9-a42d-a3dca198be8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8da2b05f-e59d-48b6-a9d7-f0b3ae44e7b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:08.125615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:08.126834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:08.128023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:08.128023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:08.128023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:08.128023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:08.128023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:08.128023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:08.128023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:08.128023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:08.128023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:08.128023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:08.128023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:08.128023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:08.128842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48149d5-04ab-4af9-a42d-a3dca198be8d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:08.134222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 325/644] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:08.134222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea1c29-3ddb-42e6-9a19-33e9baf4bc8c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:08.134222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea1c29-3ddb-42e6-9a19-33e9baf4bc8c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:08.134222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea1c29-3ddb-42e6-9a19-33e9baf4bc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:08.144947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 320/645] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:08.146009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48149d5-04ab-4af9-a42d-a3dca198be8d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:08.153278 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-dcea1c29-3ddb-42e6-9a19-33e9baf4bc8c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:08.154461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-dcea1c29-3ddb-42e6-9a19-33e9baf4bc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:08.226977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 326/646] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:08.228052 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea1c29-3ddb-42e6-9a19-33e9baf4bc8c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:08.262925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48149d5-04ab-4af9-a42d-a3dca198be8d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:08.287533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:08.303996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305ab78-9624-4fb1-aa96-1d46b68e1823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:08.304656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305ab78-9624-4fb1-aa96-1d46b68e1823 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:08.304938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305ab78-9624-4fb1-aa96-1d46b68e1823 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:08.305209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305ab78-9624-4fb1-aa96-1d46b68e1823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:08.363243 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea1c29-3ddb-42e6-9a19-33e9baf4bc8c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:08.390764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305ab78-9624-4fb1-aa96-1d46b68e1823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:08.391411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:08.392770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305ab78-9624-4fb1-aa96-1d46b68e1823 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:08.406283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf621e-9258-47ed-a130-0385675a4ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:08.407075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf621e-9258-47ed-a130-0385675a4ddf None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:08.407367 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf621e-9258-47ed-a130-0385675a4ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:08.407637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf621e-9258-47ed-a130-0385675a4ddf None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:08.408383 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf621e-9258-47ed-a130-0385675a4ddf None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:08.408684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf621e-9258-47ed-a130-0385675a4ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:08.410136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf621e-9258-47ed-a130-0385675a4ddf None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:08.419831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 327/647] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:04:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:08.420699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf621e-9258-47ed-a130-0385675a4ddf None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:08.535941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 321/648] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:04:08.536959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305ab78-9624-4fb1-aa96-1d46b68e1823 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:08.564380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf621e-9258-47ed-a130-0385675a4ddf None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:08.568248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:08.631200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f4b65-58ce-4b57-9bb8-c2edb85ea58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:08.631984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f4b65-58ce-4b57-9bb8-c2edb85ea58c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:08.632360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f4b65-58ce-4b57-9bb8-c2edb85ea58c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:08.632700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f4b65-58ce-4b57-9bb8-c2edb85ea58c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:08.634024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f4b65-58ce-4b57-9bb8-c2edb85ea58c None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=8c79367697f7451fb804b8641444f6aa) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:08.634572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f4b65-58ce-4b57-9bb8-c2edb85ea58c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:08.636149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f4b65-58ce-4b57-9bb8-c2edb85ea58c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:08.643522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 328/649] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:08 2022] GET /identity/v3/role_assignments?user.id=8c79367697f7451fb804b8641444f6aa&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:08.644350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f4b65-58ce-4b57-9bb8-c2edb85ea58c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:08.689838 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305ab78-9624-4fb1-aa96-1d46b68e1823 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:08.728069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:08.747085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000f1cc-f9c7-4966-a900-040695ff3d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:08.748168 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000f1cc-f9c7-4966-a900-040695ff3d57 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:08.748694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000f1cc-f9c7-4966-a900-040695ff3d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:08.749063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000f1cc-f9c7-4966-a900-040695ff3d57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:08.762833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000f1cc-f9c7-4966-a900-040695ff3d57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-0000f1cc-f9c7-4966-a900-040695ff3d57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:08.771264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:08.772339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:08.773232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 322/650] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:08.773964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000f1cc-f9c7-4966-a900-040695ff3d57 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:08.805902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f4b65-58ce-4b57-9bb8-c2edb85ea58c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:08.846193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a84fd-a456-4217-82c8-ab4090007a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:08.847395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a84fd-a456-4217-82c8-ab4090007a0b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:08.847885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a84fd-a456-4217-82c8-ab4090007a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:08.848384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a84fd-a456-4217-82c8-ab4090007a0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:08.867698 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a84fd-a456-4217-82c8-ab4090007a0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:08.870876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a84fd-a456-4217-82c8-ab4090007a0b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:08.875350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-4f5a84fd-a456-4217-82c8-ab4090007a0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:08.904353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:08.906189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:08.907958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:08.907958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:08.907958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:08.907958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:08.907958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 329/651] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:08.907958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a84fd-a456-4217-82c8-ab4090007a0b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:08.908631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000f1cc-f9c7-4966-a900-040695ff3d57 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:08.966520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76126769-c420-4a45-85f0-b22e2f657e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:08.967483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76126769-c420-4a45-85f0-b22e2f657e55 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:08.967761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76126769-c420-4a45-85f0-b22e2f657e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:08.968032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76126769-c420-4a45-85f0-b22e2f657e55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:08.986722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76126769-c420-4a45-85f0-b22e2f657e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:08.988199 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76126769-c420-4a45-85f0-b22e2f657e55 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-76126769-c420-4a45-85f0-b22e2f657e55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:09.009516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:09.010476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:09.011345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:09.011345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:09.011345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:09.011345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:09.011345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 323/652] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:09.012675 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76126769-c420-4a45-85f0-b22e2f657e55 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:09.063276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:09.069920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a84fd-a456-4217-82c8-ab4090007a0b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:09.111770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de8584-f815-4bbd-9a99-e2c5bd62cfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:09.112445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de8584-f815-4bbd-9a99-e2c5bd62cfd1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:09.112729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de8584-f815-4bbd-9a99-e2c5bd62cfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:09.113111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de8584-f815-4bbd-9a99-e2c5bd62cfd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:09.113911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de8584-f815-4bbd-9a99-e2c5bd62cfd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:09.114224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de8584-f815-4bbd-9a99-e2c5bd62cfd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:09.115752 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de8584-f815-4bbd-9a99-e2c5bd62cfd1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:09.125996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76126769-c420-4a45-85f0-b22e2f657e55 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:09.126450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 330/653] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:04:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:09.127371 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de8584-f815-4bbd-9a99-e2c5bd62cfd1 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:09.157670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:09.164927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0e083-9900-41e8-bc9d-aa6144a7f929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:09.165955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0e083-9900-41e8-bc9d-aa6144a7f929 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:09.166341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0e083-9900-41e8-bc9d-aa6144a7f929 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:09.166887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0e083-9900-41e8-bc9d-aa6144a7f929 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:09.168760 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0e083-9900-41e8-bc9d-aa6144a7f929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:09.169107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0e083-9900-41e8-bc9d-aa6144a7f929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:09.171235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0e083-9900-41e8-bc9d-aa6144a7f929 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:09.191114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 324/654] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:09.192422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0e083-9900-41e8-bc9d-aa6144a7f929 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:09.253422 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de8584-f815-4bbd-9a99-e2c5bd62cfd1 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:09.295904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267500e9-bc07-4cfe-984c-3b10c06a8d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:09.296933 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267500e9-bc07-4cfe-984c-3b10c06a8d3b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:09.297406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267500e9-bc07-4cfe-984c-3b10c06a8d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:09.297920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267500e9-bc07-4cfe-984c-3b10c06a8d3b None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:09.298847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267500e9-bc07-4cfe-984c-3b10c06a8d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:09.299336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267500e9-bc07-4cfe-984c-3b10c06a8d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:09.301146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267500e9-bc07-4cfe-984c-3b10c06a8d3b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:09.321321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 331/655] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:09.322443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267500e9-bc07-4cfe-984c-3b10c06a8d3b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:09.345507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0e083-9900-41e8-bc9d-aa6144a7f929 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:09.351624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:09.381868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ef8d2-0554-411a-a075-674138d3cb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:09.382665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef8d2-0554-411a-a075-674138d3cb77 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:09.382948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef8d2-0554-411a-a075-674138d3cb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:09.383293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef8d2-0554-411a-a075-674138d3cb77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:09.394821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ef8d2-0554-411a-a075-674138d3cb77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:09.396288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ef8d2-0554-411a-a075-674138d3cb77 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-5b9ef8d2-0554-411a-a075-674138d3cb77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:09.414935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:09.415869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:09.416770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:09.416770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:09.416770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:09.416770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:09.416770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 325/656] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:09.417263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ef8d2-0554-411a-a075-674138d3cb77 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:09.444190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267500e9-bc07-4cfe-984c-3b10c06a8d3b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:09.466422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:09.492632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0c837-f891-4fed-a7fb-a8196fe3bcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:09.493435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0c837-f891-4fed-a7fb-a8196fe3bcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:09.493716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0c837-f891-4fed-a7fb-a8196fe3bcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:09.494019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0c837-f891-4fed-a7fb-a8196fe3bcc5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:09.503269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0c837-f891-4fed-a7fb-a8196fe3bcc5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:09.504732 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0c837-f891-4fed-a7fb-a8196fe3bcc5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8ff0c837-f891-4fed-a7fb-a8196fe3bcc5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:09.524975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:09.526124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:09.527048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:09.527048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:09.527048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:09.527048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:09.527048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:09.527048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:09.527048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:09.527048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:09.527048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 18:04:09.527048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:09.528299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 332/657] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 18:04:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:09.528930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0c837-f891-4fed-a7fb-a8196fe3bcc5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:09.541703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ef8d2-0554-411a-a075-674138d3cb77 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:09.580698 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd44abe-569c-4d3e-9fcf-149ba5665b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:09.581116 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd44abe-569c-4d3e-9fcf-149ba5665b5e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:09.581404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd44abe-569c-4d3e-9fcf-149ba5665b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:09.581679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd44abe-569c-4d3e-9fcf-149ba5665b5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:09.589264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd44abe-569c-4d3e-9fcf-149ba5665b5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:09.590967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd44abe-569c-4d3e-9fcf-149ba5665b5e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-5fd44abe-569c-4d3e-9fcf-149ba5665b5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:09.606907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:09.607876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:09.608769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:09.608769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:09.608769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:09.608769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:09.608769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:09.608769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:09.608769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:09.608769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:09.613005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 326/658] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:09.614110 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd44abe-569c-4d3e-9fcf-149ba5665b5e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:09.654487 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:09.677268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0c837-f891-4fed-a7fb-a8196fe3bcc5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:09.722965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bea7a1-3ffe-4383-9447-1313e212b019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:09.724525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bea7a1-3ffe-4383-9447-1313e212b019 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:09.725013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bea7a1-3ffe-4383-9447-1313e212b019 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:09.725560 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bea7a1-3ffe-4383-9447-1313e212b019 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:09.727001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bea7a1-3ffe-4383-9447-1313e212b019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:09.727901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bea7a1-3ffe-4383-9447-1313e212b019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:09.729811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bea7a1-3ffe-4383-9447-1313e212b019 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:09.748240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 333/659] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:09.750101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bea7a1-3ffe-4383-9447-1313e212b019 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:09.751797 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd44abe-569c-4d3e-9fcf-149ba5665b5e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:09.756228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:09.794802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443ad0a-0107-485c-acba-556127f017c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:09.795629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443ad0a-0107-485c-acba-556127f017c0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:09.795992 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443ad0a-0107-485c-acba-556127f017c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:09.796698 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443ad0a-0107-485c-acba-556127f017c0 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:09.797243 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443ad0a-0107-485c-acba-556127f017c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:09.797624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443ad0a-0107-485c-acba-556127f017c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:09.799191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443ad0a-0107-485c-acba-556127f017c0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:09.824687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 327/660] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 18:04:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:09.826219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443ad0a-0107-485c-acba-556127f017c0 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:09.881501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bea7a1-3ffe-4383-9447-1313e212b019 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:09.919075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f349ee1-c813-44fe-a7a8-6203b032a456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:09.919867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f349ee1-c813-44fe-a7a8-6203b032a456 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:09.920218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f349ee1-c813-44fe-a7a8-6203b032a456 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:09.920567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f349ee1-c813-44fe-a7a8-6203b032a456 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:09.921431 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f349ee1-c813-44fe-a7a8-6203b032a456 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:09.921831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f349ee1-c813-44fe-a7a8-6203b032a456 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:09.923361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f349ee1-c813-44fe-a7a8-6203b032a456 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:09.944682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 334/661] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:09.944682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f349ee1-c813-44fe-a7a8-6203b032a456 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:09.947711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:09.948847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443ad0a-0107-485c-acba-556127f017c0 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:10.018086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481544a-19ef-4697-895c-0de7440b9961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:10.019439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481544a-19ef-4697-895c-0de7440b9961 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:10.019910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481544a-19ef-4697-895c-0de7440b9961 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:10.020385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481544a-19ef-4697-895c-0de7440b9961 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:10.031551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481544a-19ef-4697-895c-0de7440b9961 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:10.033830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481544a-19ef-4697-895c-0de7440b9961 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-3481544a-19ef-4697-895c-0de7440b9961 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:10.055653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:10.056871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:10.058172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:10.058172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:10.058172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:10.058172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:10.058172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:10.058172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:10.058172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:10.058172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:10.058172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 18:04:10.058172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:10.061619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 328/662] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:10.063133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481544a-19ef-4697-895c-0de7440b9961 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:10.091068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f349ee1-c813-44fe-a7a8-6203b032a456 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:10.095310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:10.141315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937004e8-b428-400a-ac1a-cb4936e6d950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:10.142005 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937004e8-b428-400a-ac1a-cb4936e6d950 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:10.142290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937004e8-b428-400a-ac1a-cb4936e6d950 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:10.142562 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937004e8-b428-400a-ac1a-cb4936e6d950 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:10.182136 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481544a-19ef-4697-895c-0de7440b9961 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:10.220787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a217d-e0ce-455f-bedb-3feaeed940d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:10.221815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a217d-e0ce-455f-bedb-3feaeed940d1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:10.222332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a217d-e0ce-455f-bedb-3feaeed940d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:10.222806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a217d-e0ce-455f-bedb-3feaeed940d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:10.223824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a217d-e0ce-455f-bedb-3feaeed940d1 None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=4a67120485f74f8bb0ff188cd3abd21c) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:10.224336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a217d-e0ce-455f-bedb-3feaeed940d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:10.225993 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a217d-e0ce-455f-bedb-3feaeed940d1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:10.233684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937004e8-b428-400a-ac1a-cb4936e6d950 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:10.235236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937004e8-b428-400a-ac1a-cb4936e6d950 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:10.237495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:10.237887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 329/663] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:10 2022] GET /identity/v3/role_assignments?user.id=4a67120485f74f8bb0ff188cd3abd21c&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:10.240453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a217d-e0ce-455f-bedb-3feaeed940d1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:10.373938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:10.382319 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 335/664] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:04:10.384212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20379b0a-e2bc-4737-9060-c1f12e7c6d5d None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:10.384754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20379b0a-e2bc-4737-9060-c1f12e7c6d5d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:10.385273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20379b0a-e2bc-4737-9060-c1f12e7c6d5d None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:10.388151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 336/665] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:10.389329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20379b0a-e2bc-4737-9060-c1f12e7c6d5d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:10.400504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a217d-e0ce-455f-bedb-3feaeed940d1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:10.447097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c7bda-dbe5-49f0-8f09-a41815499b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:10.447785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c7bda-dbe5-49f0-8f09-a41815499b18 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:10.448067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c7bda-dbe5-49f0-8f09-a41815499b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:10.448340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c7bda-dbe5-49f0-8f09-a41815499b18 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:10.449854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c7bda-dbe5-49f0-8f09-a41815499b18 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:10.450324 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c7bda-dbe5-49f0-8f09-a41815499b18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:10.451887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c7bda-dbe5-49f0-8f09-a41815499b18 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:10.474916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 330/666] 10.222.0.46 () {56 vars in 1139 bytes} [Thu May 26 18:04:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:10.476412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c9c91-6906-49f9-853e-577e3b6f3729 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:10.476700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c9c91-6906-49f9-853e-577e3b6f3729 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:10.477021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c9c91-6906-49f9-853e-577e3b6f3729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:10.498708 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-729c9c91-6906-49f9-853e-577e3b6f3729 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:10.499899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-729c9c91-6906-49f9-853e-577e3b6f3729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:10.557627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20379b0a-e2bc-4737-9060-c1f12e7c6d5d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:10.562210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:10.597172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 331/667] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:10.598176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c9c91-6906-49f9-853e-577e3b6f3729 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:10.609589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf53c7-93bc-4016-8003-4e62990d71a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:10.610437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf53c7-93bc-4016-8003-4e62990d71a5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:10.610716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf53c7-93bc-4016-8003-4e62990d71a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:10.610986 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf53c7-93bc-4016-8003-4e62990d71a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:10.623267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf53c7-93bc-4016-8003-4e62990d71a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:10.624746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf53c7-93bc-4016-8003-4e62990d71a5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-0caf53c7-93bc-4016-8003-4e62990d71a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:10.648092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:10.649044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:10.650001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:10.650001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:10.650001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:10.650001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:10.650001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 337/668] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:10.650891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf53c7-93bc-4016-8003-4e62990d71a5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:10.708012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:10.745413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c9c91-6906-49f9-853e-577e3b6f3729 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:10.774526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf53c7-93bc-4016-8003-4e62990d71a5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:10.795235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479a1ce-8189-427b-a1c9-6e4654d29e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:10.796191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479a1ce-8189-427b-a1c9-6e4654d29e1b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:10.797222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479a1ce-8189-427b-a1c9-6e4654d29e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:10.797710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479a1ce-8189-427b-a1c9-6e4654d29e1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:10.812643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57bb5b-7b3e-4d44-8cc0-dd3d6ba4ab80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:10.813705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57bb5b-7b3e-4d44-8cc0-dd3d6ba4ab80 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:10.813848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57bb5b-7b3e-4d44-8cc0-dd3d6ba4ab80 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:10.814172 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57bb5b-7b3e-4d44-8cc0-dd3d6ba4ab80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:10.826163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57bb5b-7b3e-4d44-8cc0-dd3d6ba4ab80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-3f57bb5b-7b3e-4d44-8cc0-dd3d6ba4ab80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:10.834270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:10.835232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:10.836056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 338/669] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:10.837497 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57bb5b-7b3e-4d44-8cc0-dd3d6ba4ab80 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:10.856936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:10.895340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479a1ce-8189-427b-a1c9-6e4654d29e1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:10.897871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479a1ce-8189-427b-a1c9-6e4654d29e1b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:10.979661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57bb5b-7b3e-4d44-8cc0-dd3d6ba4ab80 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:10.992415 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:11.022706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 332/670] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:04:11.023615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479a1ce-8189-427b-a1c9-6e4654d29e1b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:11.029852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71be363-cee4-41ad-8572-1634cf366e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:11.030795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71be363-cee4-41ad-8572-1634cf366e79 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:11.031255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71be363-cee4-41ad-8572-1634cf366e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:11.031728 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71be363-cee4-41ad-8572-1634cf366e79 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:11.032629 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71be363-cee4-41ad-8572-1634cf366e79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:11.033111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71be363-cee4-41ad-8572-1634cf366e79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:11.035257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71be363-cee4-41ad-8572-1634cf366e79 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:11.054171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 339/671] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:11.055354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71be363-cee4-41ad-8572-1634cf366e79 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:11.171292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:11.173009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479a1ce-8189-427b-a1c9-6e4654d29e1b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:11.211034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dede925-dfe3-4d08-927b-1bfee5c9dff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:11.211448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71be363-cee4-41ad-8572-1634cf366e79 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:11.211706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dede925-dfe3-4d08-927b-1bfee5c9dff8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:11.211987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dede925-dfe3-4d08-927b-1bfee5c9dff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:11.212258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dede925-dfe3-4d08-927b-1bfee5c9dff8 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:11.213001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dede925-dfe3-4d08-927b-1bfee5c9dff8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:11.213304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dede925-dfe3-4d08-927b-1bfee5c9dff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:11.214740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dede925-dfe3-4d08-927b-1bfee5c9dff8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:11.224946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 333/672] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:04:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:11.225844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dede925-dfe3-4d08-927b-1bfee5c9dff8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:11.256628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfafaf-118b-4318-9aef-905328bab3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:11.257852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfafaf-118b-4318-9aef-905328bab3cf None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:11.258138 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfafaf-118b-4318-9aef-905328bab3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:11.258406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfafaf-118b-4318-9aef-905328bab3cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:11.268491 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:11.280419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfafaf-118b-4318-9aef-905328bab3cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:11.282095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfafaf-118b-4318-9aef-905328bab3cf None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-22cfafaf-118b-4318-9aef-905328bab3cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:11.304085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:11.305139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:11.306059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:11.306059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:11.306059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:11.306059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:11.306059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 340/673] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:11.306752 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bdf95-7aab-4150-b43a-ff84303ecd2c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:11.306929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bdf95-7aab-4150-b43a-ff84303ecd2c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:11.307203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bdf95-7aab-4150-b43a-ff84303ecd2c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:11.313511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 341/674] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:11.313511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bdf95-7aab-4150-b43a-ff84303ecd2c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:11.352632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dede925-dfe3-4d08-927b-1bfee5c9dff8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:11.390195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c59210-0ddb-41d0-a818-83799bb49c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:11.390979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c59210-0ddb-41d0-a818-83799bb49c49 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:11.391258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c59210-0ddb-41d0-a818-83799bb49c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:11.391526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c59210-0ddb-41d0-a818-83799bb49c49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:11.398153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c59210-0ddb-41d0-a818-83799bb49c49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:11.399575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c59210-0ddb-41d0-a818-83799bb49c49 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-33c59210-0ddb-41d0-a818-83799bb49c49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:11.414240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:11.415114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:11.415997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:11.415997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:11.415997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:11.415997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:11.415997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:11.415997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:11.415997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:11.415997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:11.415997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 334/675] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:11.416465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c59210-0ddb-41d0-a818-83799bb49c49 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:11.431764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bdf95-7aab-4150-b43a-ff84303ecd2c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:11.470385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:11.484829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b38567-e53a-4b75-8a19-deeee037f94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:11.485742 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b38567-e53a-4b75-8a19-deeee037f94b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:11.486030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b38567-e53a-4b75-8a19-deeee037f94b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:11.486293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b38567-e53a-4b75-8a19-deeee037f94b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:11.498294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b38567-e53a-4b75-8a19-deeee037f94b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:11.499825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b38567-e53a-4b75-8a19-deeee037f94b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:11.517787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-51b38567-e53a-4b75-8a19-deeee037f94b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:11.520564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:11.521462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:11.522399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:11.522399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:11.522399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:11.522399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:11.522615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 342/676] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:11.523787 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd35a0-06bd-400d-a476-58af2418d1d5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:11.524078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd35a0-06bd-400d-a476-58af2418d1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:11.524353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd35a0-06bd-400d-a476-58af2418d1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:11.529999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c59210-0ddb-41d0-a818-83799bb49c49 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:11.541316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f7dd35a0-06bd-400d-a476-58af2418d1d5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:11.542575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f7dd35a0-06bd-400d-a476-58af2418d1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:11.568866 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed16bd-c03f-4666-8ebe-54ebd7c8c5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:11.569526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed16bd-c03f-4666-8ebe-54ebd7c8c5ae None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:11.569840 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed16bd-c03f-4666-8ebe-54ebd7c8c5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:11.570177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed16bd-c03f-4666-8ebe-54ebd7c8c5ae None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:11.570826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed16bd-c03f-4666-8ebe-54ebd7c8c5ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:11.571129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed16bd-c03f-4666-8ebe-54ebd7c8c5ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:11.572652 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed16bd-c03f-4666-8ebe-54ebd7c8c5ae None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:11.589890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 335/677] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:11.589890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed16bd-c03f-4666-8ebe-54ebd7c8c5ae None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:11.620330 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 343/678] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:11.622107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd35a0-06bd-400d-a476-58af2418d1d5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:11.711376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed16bd-c03f-4666-8ebe-54ebd7c8c5ae None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:11.731854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd35a0-06bd-400d-a476-58af2418d1d5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:11.741587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:11.749592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08833863-6aab-430c-893e-67704b8c469d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:11.750304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08833863-6aab-430c-893e-67704b8c469d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:11.750587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08833863-6aab-430c-893e-67704b8c469d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:11.750860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08833863-6aab-430c-893e-67704b8c469d None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:11.751762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08833863-6aab-430c-893e-67704b8c469d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:11.752075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08833863-6aab-430c-893e-67704b8c469d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:11.753510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08833863-6aab-430c-893e-67704b8c469d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:11.770172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 336/679] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:11.770172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08833863-6aab-430c-893e-67704b8c469d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:11.775834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316555d-7c04-47a1-832b-b94a5a42c2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:11.776502 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316555d-7c04-47a1-832b-b94a5a42c2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:11.776784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316555d-7c04-47a1-832b-b94a5a42c2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:11.777057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316555d-7c04-47a1-832b-b94a5a42c2f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:11.777840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316555d-7c04-47a1-832b-b94a5a42c2f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:11.778147 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316555d-7c04-47a1-832b-b94a5a42c2f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:11.779548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316555d-7c04-47a1-832b-b94a5a42c2f0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:11.789511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:11.797193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 344/680] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:11.797193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316555d-7c04-47a1-832b-b94a5a42c2f0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:11.879182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08833863-6aab-430c-893e-67704b8c469d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:11.907916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316555d-7c04-47a1-832b-b94a5a42c2f0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:11.916532 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93de4ac-c4f8-4c44-837c-1ebea8590f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:11.917321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93de4ac-c4f8-4c44-837c-1ebea8590f42 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:11.917599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93de4ac-c4f8-4c44-837c-1ebea8590f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:11.918969 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93de4ac-c4f8-4c44-837c-1ebea8590f42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:11.926612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93de4ac-c4f8-4c44-837c-1ebea8590f42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:11.927818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93de4ac-c4f8-4c44-837c-1ebea8590f42 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e93de4ac-c4f8-4c44-837c-1ebea8590f42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:11.942095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:11.943126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:11.944040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:11.944040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:11.944040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:11.944040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:11.944040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:11.944040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:11.944040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:11.944040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:11.946190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef8c9e-a76b-4e25-bb40-135bada78b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:11.947003 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef8c9e-a76b-4e25-bb40-135bada78b48 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:11.947286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef8c9e-a76b-4e25-bb40-135bada78b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:11.947770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef8c9e-a76b-4e25-bb40-135bada78b48 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:11.947871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 337/681] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:11.948368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef8c9e-a76b-4e25-bb40-135bada78b48 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:11.948714 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93de4ac-c4f8-4c44-837c-1ebea8590f42 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:11.950581 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef8c9e-a76b-4e25-bb40-135bada78b48 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-09ef8c9e-a76b-4e25-bb40-135bada78b48 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:11.965960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:11.967199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:11.969380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 345/682] 10.222.0.46 () {56 vars in 1129 bytes} [Thu May 26 18:04:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:11.971616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef8c9e-a76b-4e25-bb40-135bada78b48 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:12.004529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:12.039006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:12.064590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93de4ac-c4f8-4c44-837c-1ebea8590f42 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:12.089061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef8c9e-a76b-4e25-bb40-135bada78b48 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:12.099969 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb5800-6708-4b20-b8cc-606595199095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:12.100886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb5800-6708-4b20-b8cc-606595199095 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:12.101030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb5800-6708-4b20-b8cc-606595199095 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:12.101237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb5800-6708-4b20-b8cc-606595199095 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:12.102077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb5800-6708-4b20-b8cc-606595199095 None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=d420e808730d484f80963322c5fc6856) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:12.102386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb5800-6708-4b20-b8cc-606595199095 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:12.103798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb5800-6708-4b20-b8cc-606595199095 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:12.117773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 338/683] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:11 2022] GET /identity/v3/role_assignments?user.id=d420e808730d484f80963322c5fc6856&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:12.117773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb5800-6708-4b20-b8cc-606595199095 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:12.135836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64903a14-9f3e-480a-88f9-66869ae059ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:12.137657 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64903a14-9f3e-480a-88f9-66869ae059ce None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:12.138324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64903a14-9f3e-480a-88f9-66869ae059ce None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:12.138719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64903a14-9f3e-480a-88f9-66869ae059ce None admin] PATH_INFO: `/v3/users/swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:12.148191 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64903a14-9f3e-480a-88f9-66869ae059ce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:12.149885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64903a14-9f3e-480a-88f9-66869ae059ce None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-64903a14-9f3e-480a-88f9-66869ae059ce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:12.166746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:12.168456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:12.169601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:12.169601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:12.169601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:12.169601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:12.169601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:12.169601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:12.169601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:12.169601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:12.169601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 18:04:12.169601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:12.169601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 346/684] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:04:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:12.169601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64903a14-9f3e-480a-88f9-66869ae059ce None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:12.256378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb5800-6708-4b20-b8cc-606595199095 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:12.303554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b681e-f423-4bd4-bc5c-5d15a550d7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:12.304381 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b681e-f423-4bd4-bc5c-5d15a550d7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:12.304855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b681e-f423-4bd4-bc5c-5d15a550d7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:12.305225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b681e-f423-4bd4-bc5c-5d15a550d7b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:12.306183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b681e-f423-4bd4-bc5c-5d15a550d7b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:12.306590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b681e-f423-4bd4-bc5c-5d15a550d7b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:12.308252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b681e-f423-4bd4-bc5c-5d15a550d7b2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:12.309336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:12.322820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:12.333223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 339/685] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:12.333223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b681e-f423-4bd4-bc5c-5d15a550d7b2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:12.334214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64903a14-9f3e-480a-88f9-66869ae059ce None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:12.380244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbb813-e5d9-43b1-af73-f866ea1186dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:12.381175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbb813-e5d9-43b1-af73-f866ea1186dd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:12.381603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbb813-e5d9-43b1-af73-f866ea1186dd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:12.382043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbb813-e5d9-43b1-af73-f866ea1186dd None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:12.382910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbb813-e5d9-43b1-af73-f866ea1186dd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:12.383418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbb813-e5d9-43b1-af73-f866ea1186dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:12.384989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbb813-e5d9-43b1-af73-f866ea1186dd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:12.393004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 347/686] 10.222.0.46 () {56 vars in 1124 bytes} [Thu May 26 18:04:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:12.397843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbb813-e5d9-43b1-af73-f866ea1186dd None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:12.454471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b681e-f423-4bd4-bc5c-5d15a550d7b2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:12.510118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296794-1f1f-416f-b9c4-167c72eb4efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:12.510875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296794-1f1f-416f-b9c4-167c72eb4efc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:12.511161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296794-1f1f-416f-b9c4-167c72eb4efc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:12.511437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296794-1f1f-416f-b9c4-167c72eb4efc None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:12.513118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb296794-1f1f-416f-b9c4-167c72eb4efc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:12.513512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb296794-1f1f-416f-b9c4-167c72eb4efc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:12.515500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb296794-1f1f-416f-b9c4-167c72eb4efc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:12.537114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 340/687] 10.222.0.46 () {56 vars in 1135 bytes} [Thu May 26 18:04:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:12.538166 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296794-1f1f-416f-b9c4-167c72eb4efc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:12.547312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbb813-e5d9-43b1-af73-f866ea1186dd None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:12.584064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfb884-7db9-4d6a-a3e4-24d57728e284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:12.585060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfb884-7db9-4d6a-a3e4-24d57728e284 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:12.585536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfb884-7db9-4d6a-a3e4-24d57728e284 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:12.586036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfb884-7db9-4d6a-a3e4-24d57728e284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:12.587067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfb884-7db9-4d6a-a3e4-24d57728e284 None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=f668c4c6745e4f1eae20d8a3a12cf0cc) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:12.587573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfb884-7db9-4d6a-a3e4-24d57728e284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:12.589208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfb884-7db9-4d6a-a3e4-24d57728e284 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:12.597301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 348/688] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:12 2022] GET /identity/v3/role_assignments?user.id=f668c4c6745e4f1eae20d8a3a12cf0cc&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:12.602138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:12.602864 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfb884-7db9-4d6a-a3e4-24d57728e284 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:12.640180 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:12.662764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296794-1f1f-416f-b9c4-167c72eb4efc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:12.714705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe440e2-ad85-4f2e-9a31-5f580a5df962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:12.715917 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe440e2-ad85-4f2e-9a31-5f580a5df962 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:12.716197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe440e2-ad85-4f2e-9a31-5f580a5df962 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:12.716472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe440e2-ad85-4f2e-9a31-5f580a5df962 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:12.717204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe440e2-ad85-4f2e-9a31-5f580a5df962 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:12.717514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe440e2-ad85-4f2e-9a31-5f580a5df962 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:12.718995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe440e2-ad85-4f2e-9a31-5f580a5df962 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:12.729832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 341/689] 10.222.0.46 () {56 vars in 1124 bytes} [Thu May 26 18:04:12 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:12.729832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeed96c-c4b9-45c3-a89d-78959bd3eaa8 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:12.729832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeed96c-c4b9-45c3-a89d-78959bd3eaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:12.729832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeed96c-c4b9-45c3-a89d-78959bd3eaa8 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:12.730235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 342/690] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:12.737456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246d3d4-3dc0-4fb0-b441-e856f26d2ba1 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:12.737456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246d3d4-3dc0-4fb0-b441-e856f26d2ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:12.737456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246d3d4-3dc0-4fb0-b441-e856f26d2ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:12.745328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfb884-7db9-4d6a-a3e4-24d57728e284 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:12.772657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e246d3d4-3dc0-4fb0-b441-e856f26d2ba1 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:12.776067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e246d3d4-3dc0-4fb0-b441-e856f26d2ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:12.859319 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1efcc-1b79-4c25-bed1-a0a64dd1b3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:12.859901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1efcc-1b79-4c25-bed1-a0a64dd1b3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:12.860183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1efcc-1b79-4c25-bed1-a0a64dd1b3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:12.860456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1efcc-1b79-4c25-bed1-a0a64dd1b3e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:12.885624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 343/691] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:12.911858 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246d3d4-3dc0-4fb0-b441-e856f26d2ba1 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:12.956662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1efcc-1b79-4c25-bed1-a0a64dd1b3e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:12.958249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1efcc-1b79-4c25-bed1-a0a64dd1b3e7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:12.979006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:12.986127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:13.071473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246d3d4-3dc0-4fb0-b441-e856f26d2ba1 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:13.130989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f21232-5c49-42b0-945d-2e3eb22a8a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:13.131819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f21232-5c49-42b0-945d-2e3eb22a8a73 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:13.132107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f21232-5c49-42b0-945d-2e3eb22a8a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:13.132382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f21232-5c49-42b0-945d-2e3eb22a8a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:13.144573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f21232-5c49-42b0-945d-2e3eb22a8a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:13.146070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f21232-5c49-42b0-945d-2e3eb22a8a73 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:13.156713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 349/692] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:04:13.164258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1efcc-1b79-4c25-bed1-a0a64dd1b3e7 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-15f21232-5c49-42b0-945d-2e3eb22a8a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:13.171103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:13.171985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:13.172853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:13.172853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:13.172853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:13.172853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:13.172853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 344/693] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:13.178794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f21232-5c49-42b0-945d-2e3eb22a8a73 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:13.378244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f21232-5c49-42b0-945d-2e3eb22a8a73 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:13.387414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:13.462851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fbf56-c815-4c43-903d-789858f68bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:13.463503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fbf56-c815-4c43-903d-789858f68bce None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:13.463779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fbf56-c815-4c43-903d-789858f68bce None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:13.464046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fbf56-c815-4c43-903d-789858f68bce None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:13.464867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fbf56-c815-4c43-903d-789858f68bce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:13.465641 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fbf56-c815-4c43-903d-789858f68bce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:13.467199 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fbf56-c815-4c43-903d-789858f68bce None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:13.487911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 345/694] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:13.493412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fbf56-c815-4c43-903d-789858f68bce None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:13.515739 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1efcc-1b79-4c25-bed1-a0a64dd1b3e7 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:13.558020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:13.602465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fbf56-c815-4c43-903d-789858f68bce None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:13.603464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e2427-fc41-41a9-a332-e6b8a7ea95b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:13.604272 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e2427-fc41-41a9-a332-e6b8a7ea95b8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:13.604555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e2427-fc41-41a9-a332-e6b8a7ea95b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:13.604827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e2427-fc41-41a9-a332-e6b8a7ea95b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:13.628066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e2427-fc41-41a9-a332-e6b8a7ea95b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:13.629819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e2427-fc41-41a9-a332-e6b8a7ea95b8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:13.640097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ed00c-f969-49af-8a14-50ec2578a39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:13.640698 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ed00c-f969-49af-8a14-50ec2578a39b None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:13.640979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ed00c-f969-49af-8a14-50ec2578a39b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:13.641251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ed00c-f969-49af-8a14-50ec2578a39b None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:13.641990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ed00c-f969-49af-8a14-50ec2578a39b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:13.643420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ed00c-f969-49af-8a14-50ec2578a39b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8e7e2427-fc41-41a9-a332-e6b8a7ea95b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:13.663026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:13.663967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:13.664846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:13.664846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:13.664846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:13.664846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:13.664846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 350/695] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 501 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:13.670462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e2427-fc41-41a9-a332-e6b8a7ea95b8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:13.672281 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 346/696] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:04:13 2022] POST /identity/v3/users => generated 254 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:13.810710 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e2427-fc41-41a9-a332-e6b8a7ea95b8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:13.852842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ec043-49c4-4cec-a0e5-653a92099e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:13.853516 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ec043-49c4-4cec-a0e5-653a92099e53 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:13.853840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ec043-49c4-4cec-a0e5-653a92099e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:13.854114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ec043-49c4-4cec-a0e5-653a92099e53 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:13.854902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ec043-49c4-4cec-a0e5-653a92099e53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:13.855218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ec043-49c4-4cec-a0e5-653a92099e53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:13.857546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ec043-49c4-4cec-a0e5-653a92099e53 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:13.870869 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 351/697] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:13.876811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ed00c-f969-49af-8a14-50ec2578a39b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:13.923204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:14.016258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ed00c-f969-49af-8a14-50ec2578a39b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:14.055945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7404c-ce74-4ac7-97dd-fb9f384ac963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:14.056992 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7404c-ce74-4ac7-97dd-fb9f384ac963 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:14.057416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7404c-ce74-4ac7-97dd-fb9f384ac963 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:14.057877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7404c-ce74-4ac7-97dd-fb9f384ac963 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:14.065365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7404c-ce74-4ac7-97dd-fb9f384ac963 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:14.066992 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7404c-ce74-4ac7-97dd-fb9f384ac963 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e9d7404c-ce74-4ac7-97dd-fb9f384ac963 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:14.083653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:14.084757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:14.085854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:14.085854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:14.085854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:14.085854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:14.085854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:14.085854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:14.085854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:14.085854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:14.087944 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 347/698] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:14.094079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ec043-49c4-4cec-a0e5-653a92099e53 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:14.139537 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:14.227246 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ec043-49c4-4cec-a0e5-653a92099e53 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:14.279438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cf166-2da3-4593-adae-ba94a1a1b8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:14.280467 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cf166-2da3-4593-adae-ba94a1a1b8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:14.280957 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cf166-2da3-4593-adae-ba94a1a1b8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:14.281437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cf166-2da3-4593-adae-ba94a1a1b8b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:14.282503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cf166-2da3-4593-adae-ba94a1a1b8b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:14.283022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cf166-2da3-4593-adae-ba94a1a1b8b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:14.284675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cf166-2da3-4593-adae-ba94a1a1b8b9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:14.298616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 352/699] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:14.304604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7404c-ce74-4ac7-97dd-fb9f384ac963 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:14.436858 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:14.462067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7404c-ce74-4ac7-97dd-fb9f384ac963 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:14.514211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5b741-89f9-4bbb-8847-0defef161151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:14.514809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5b741-89f9-4bbb-8847-0defef161151 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:14.515062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5b741-89f9-4bbb-8847-0defef161151 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:14.515321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5b741-89f9-4bbb-8847-0defef161151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:14.516144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5b741-89f9-4bbb-8847-0defef161151 None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=d656b42c2acb498caa92c77a22fd7efe) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:14.516457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5b741-89f9-4bbb-8847-0defef161151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:14.517912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5b741-89f9-4bbb-8847-0defef161151 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:14.530869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 348/700] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:14 2022] GET /identity/v3/role_assignments?user.id=d656b42c2acb498caa92c77a22fd7efe&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:14.959537 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc65c14-66e7-4582-a4d5-7db8cdbdebff None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:14.960050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc65c14-66e7-4582-a4d5-7db8cdbdebff None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:14.960457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc65c14-66e7-4582-a4d5-7db8cdbdebff None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:14.961850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 353/701] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:14.968447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b4e28-8a1a-43ae-a4d6-f40305fed2f1 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:14.968738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b4e28-8a1a-43ae-a4d6-f40305fed2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:14.969017 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b4e28-8a1a-43ae-a4d6-f40305fed2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:14.988454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a03b4e28-8a1a-43ae-a4d6-f40305fed2f1 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:14.989655 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a03b4e28-8a1a-43ae-a4d6-f40305fed2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:15.087463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 349/702] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:15.113536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc65c14-66e7-4582-a4d5-7db8cdbdebff None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:15.242520 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:15.259665 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc65c14-66e7-4582-a4d5-7db8cdbdebff None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:15.324990 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c4207-a006-4f7d-9e94-46b1c9ca2c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:15.324990 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c4207-a006-4f7d-9e94-46b1c9ca2c42 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:15.324990 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c4207-a006-4f7d-9e94-46b1c9ca2c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:15.324990 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c4207-a006-4f7d-9e94-46b1c9ca2c42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:15.341369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c4207-a006-4f7d-9e94-46b1c9ca2c42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9e5c4207-a006-4f7d-9e94-46b1c9ca2c42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:15.352725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:15.353770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:15.354617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 354/703] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:04:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:15.361870 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b4e28-8a1a-43ae-a4d6-f40305fed2f1 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:15.442654 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da395ed0-dd7f-4466-86ac-4418eaf3fbd2 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:15.443397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da395ed0-dd7f-4466-86ac-4418eaf3fbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:15.443899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da395ed0-dd7f-4466-86ac-4418eaf3fbd2 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:15.446977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 355/704] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:15.452822 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d11c4-15f4-4a20-a911-e388f556abaa None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:15.453395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d11c4-15f4-4a20-a911-e388f556abaa None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:15.453912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d11c4-15f4-4a20-a911-e388f556abaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:15.476146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a81d11c4-15f4-4a20-a911-e388f556abaa None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:15.477819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a81d11c4-15f4-4a20-a911-e388f556abaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:15.490581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:15.559286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 356/705] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 18:04:15.566702 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b4e28-8a1a-43ae-a4d6-f40305fed2f1 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:15.585189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d11c4-15f4-4a20-a911-e388f556abaa None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:15.619830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c399f43-1c63-4770-92a4-3b3e5059c744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:15.620515 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c399f43-1c63-4770-92a4-3b3e5059c744 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:15.620796 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c399f43-1c63-4770-92a4-3b3e5059c744 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:15.621105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c399f43-1c63-4770-92a4-3b3e5059c744 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:15.621890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c399f43-1c63-4770-92a4-3b3e5059c744 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:15.622203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c399f43-1c63-4770-92a4-3b3e5059c744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:15.623601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c399f43-1c63-4770-92a4-3b3e5059c744 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:15.634876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 350/706] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:04:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:15.636332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f78e3-0f5c-4d8e-9832-1c770f167a1e None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:15.637182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f78e3-0f5c-4d8e-9832-1c770f167a1e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:15.637539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f78e3-0f5c-4d8e-9832-1c770f167a1e None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:15.644345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 351/707] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:15.645197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f78e3-0f5c-4d8e-9832-1c770f167a1e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:15.719911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:15.745747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d11c4-15f4-4a20-a911-e388f556abaa None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:15.788979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f78e3-0f5c-4d8e-9832-1c770f167a1e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:15.798391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7e433-b316-4e9e-a753-14de13ed8343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:15.799226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7e433-b316-4e9e-a753-14de13ed8343 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:15.799613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7e433-b316-4e9e-a753-14de13ed8343 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:15.799989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7e433-b316-4e9e-a753-14de13ed8343 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:15.800947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7e433-b316-4e9e-a753-14de13ed8343 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:15.803091 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7e433-b316-4e9e-a753-14de13ed8343 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:15.832899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 357/708] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:04:15 2022] POST /identity/v3/services => generated 251 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:04:15.835349 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f763b3f-6487-4551-961d-416ea088fc37 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:15.836066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f763b3f-6487-4551-961d-416ea088fc37 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:15.836599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f763b3f-6487-4551-961d-416ea088fc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:15.859054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3f763b3f-6487-4551-961d-416ea088fc37 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:15.861351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3f763b3f-6487-4551-961d-416ea088fc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:15.917561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:15.938587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75789b-4636-486b-9400-36e1f7cb83cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:15.939394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75789b-4636-486b-9400-36e1f7cb83cc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:15.939679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75789b-4636-486b-9400-36e1f7cb83cc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:15.939970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75789b-4636-486b-9400-36e1f7cb83cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:15.947098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 358/709] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:15.967310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75789b-4636-486b-9400-36e1f7cb83cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:15.968812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75789b-4636-486b-9400-36e1f7cb83cc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:15.987411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f763b3f-6487-4551-961d-416ea088fc37 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-8c75789b-4636-486b-9400-36e1f7cb83cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:15.990529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:15.991473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:15.992404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:15.992404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:15.992404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:15.992404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:15.994058 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 352/710] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:16.006964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75789b-4636-486b-9400-36e1f7cb83cc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:16.079688 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:16.117763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f763b3f-6487-4551-961d-416ea088fc37 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:16.162612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751167e7-183c-452c-83c3-1d3d5c847102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:16.165184 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751167e7-183c-452c-83c3-1d3d5c847102 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:16.165654 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751167e7-183c-452c-83c3-1d3d5c847102 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:16.166148 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751167e7-183c-452c-83c3-1d3d5c847102 None admin] PATH_INFO: `/v3/services/network` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:16.166952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751167e7-183c-452c-83c3-1d3d5c847102 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:16.168653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751167e7-183c-452c-83c3-1d3d5c847102 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-751167e7-183c-452c-83c3-1d3d5c847102 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:16.184660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:16.185968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 18:04:16.187059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:16.193922 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 359/711] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:15 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:16.196771 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751167e7-183c-452c-83c3-1d3d5c847102 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:16.218326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75789b-4636-486b-9400-36e1f7cb83cc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:16.259681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea276802-35bd-4d07-8da8-2275abe97db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:16.260649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea276802-35bd-4d07-8da8-2275abe97db5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:16.260764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea276802-35bd-4d07-8da8-2275abe97db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:16.260994 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea276802-35bd-4d07-8da8-2275abe97db5 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:16.262301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea276802-35bd-4d07-8da8-2275abe97db5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:16.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea276802-35bd-4d07-8da8-2275abe97db5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:16.264189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea276802-35bd-4d07-8da8-2275abe97db5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:16.282170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 353/712] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:16.289120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea276802-35bd-4d07-8da8-2275abe97db5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:16.333458 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751167e7-183c-452c-83c3-1d3d5c847102 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:16.356478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:16.381037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dbb27-97dc-4ef4-b39c-4019836c4d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:16.381631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dbb27-97dc-4ef4-b39c-4019836c4d66 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:16.382611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dbb27-97dc-4ef4-b39c-4019836c4d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:16.382900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dbb27-97dc-4ef4-b39c-4019836c4d66 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:16.383608 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78dbb27-97dc-4ef4-b39c-4019836c4d66 None admin] RBAC: Adding query filter params (name=network) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:16.383916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78dbb27-97dc-4ef4-b39c-4019836c4d66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:16.385297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78dbb27-97dc-4ef4-b39c-4019836c4d66 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:16.387585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:16.391168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 360/713] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:16.398777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dbb27-97dc-4ef4-b39c-4019836c4d66 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:16.479981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea276802-35bd-4d07-8da8-2275abe97db5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:16.517617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dbb27-97dc-4ef4-b39c-4019836c4d66 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:16.566241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270089f-15de-4bf2-b76f-a6c3d0623abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:16.568277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270089f-15de-4bf2-b76f-a6c3d0623abc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:16.569879 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270089f-15de-4bf2-b76f-a6c3d0623abc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:16.570405 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270089f-15de-4bf2-b76f-a6c3d0623abc None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:16.571429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270089f-15de-4bf2-b76f-a6c3d0623abc None admin] RBAC: Adding query filter params (type=network) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:16.572695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270089f-15de-4bf2-b76f-a6c3d0623abc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:16.575030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270089f-15de-4bf2-b76f-a6c3d0623abc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:16.583006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c1097-5317-41b1-babc-9b192dc9b99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:16.584386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c1097-5317-41b1-babc-9b192dc9b99e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:16.584844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c1097-5317-41b1-babc-9b192dc9b99e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:16.585292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c1097-5317-41b1-babc-9b192dc9b99e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:16.591100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 361/714] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:16 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:16.602977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c1097-5317-41b1-babc-9b192dc9b99e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:16.604836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c1097-5317-41b1-babc-9b192dc9b99e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-321c1097-5317-41b1-babc-9b192dc9b99e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:16.627038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:16.628045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:16.629010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:16.629010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:16.629010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:16.629010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:16.629010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:16.629010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:16.629010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:16.629010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:16.629010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 18:04:16.629010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:16.631302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 354/715] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 18:04:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:16.640131 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270089f-15de-4bf2-b76f-a6c3d0623abc None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:16.716162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24d24f-198e-4183-bcd6-7f26d026c9b6 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:16.716533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24d24f-198e-4183-bcd6-7f26d026c9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:16.716715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24d24f-198e-4183-bcd6-7f26d026c9b6 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:16.718788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 355/716] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:16.724395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:16.726618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac103353-77d5-4915-9590-039fadfb2c98 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:16.727124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac103353-77d5-4915-9590-039fadfb2c98 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:16.727573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac103353-77d5-4915-9590-039fadfb2c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:16.763233 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ac103353-77d5-4915-9590-039fadfb2c98 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:16.767028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ac103353-77d5-4915-9590-039fadfb2c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:16.791464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270089f-15de-4bf2-b76f-a6c3d0623abc None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:16.833489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ce77d-3910-464e-874a-8890a79a3c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:16.834515 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce77d-3910-464e-874a-8890a79a3c0d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:16.834975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce77d-3910-464e-874a-8890a79a3c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:16.835445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce77d-3910-464e-874a-8890a79a3c0d None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:16.836480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ce77d-3910-464e-874a-8890a79a3c0d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:16.836975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ce77d-3910-464e-874a-8890a79a3c0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:16.838673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ce77d-3910-464e-874a-8890a79a3c0d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:16.858592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 362/717] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 18:04:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:16.865339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ce77d-3910-464e-874a-8890a79a3c0d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:16.871237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 356/718] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:16.903329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac103353-77d5-4915-9590-039fadfb2c98 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:16.942814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:16.984192 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ce77d-3910-464e-874a-8890a79a3c0d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:17.012934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:17.026211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d0bcf-64f4-453d-8381-f4a3879c1972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:17.027250 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d0bcf-64f4-453d-8381-f4a3879c1972 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:17.028277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d0bcf-64f4-453d-8381-f4a3879c1972 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:17.028769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d0bcf-64f4-453d-8381-f4a3879c1972 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:17.078444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac103353-77d5-4915-9590-039fadfb2c98 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:17.101748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d0bcf-64f4-453d-8381-f4a3879c1972 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:17.103777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d0bcf-64f4-453d-8381-f4a3879c1972 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:17.159706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62c457-50d7-4eb3-99b4-0da14426f14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:17.160583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62c457-50d7-4eb3-99b4-0da14426f14d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:17.161310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62c457-50d7-4eb3-99b4-0da14426f14d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:17.161586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62c457-50d7-4eb3-99b4-0da14426f14d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:17.187560 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62c457-50d7-4eb3-99b4-0da14426f14d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-fc62c457-50d7-4eb3-99b4-0da14426f14d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:17.215628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:17.216855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:17.222352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 357/719] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:17.226903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 363/720] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:16 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 18:04:17.234811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62c457-50d7-4eb3-99b4-0da14426f14d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:17.236005 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d0bcf-64f4-453d-8381-f4a3879c1972 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:17.298385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:17.451013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d0bcf-64f4-453d-8381-f4a3879c1972 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:17.494746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e829e9-c96f-43ac-ab76-f39cb95fbf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:17.495590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e829e9-c96f-43ac-ab76-f39cb95fbf86 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:17.496718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e829e9-c96f-43ac-ab76-f39cb95fbf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:17.497125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e829e9-c96f-43ac-ab76-f39cb95fbf86 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:17.497945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e829e9-c96f-43ac-ab76-f39cb95fbf86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:17.498255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e829e9-c96f-43ac-ab76-f39cb95fbf86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:17.499824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e829e9-c96f-43ac-ab76-f39cb95fbf86 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:17.515312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 364/721] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:04:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:17.517649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e829e9-c96f-43ac-ab76-f39cb95fbf86 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:17.532053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:17.560795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62c457-50d7-4eb3-99b4-0da14426f14d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:17.654125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:17.668921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e829e9-c96f-43ac-ab76-f39cb95fbf86 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:17.707050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0639f-59be-471b-9562-5f51c039ef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:17.707960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0639f-59be-471b-9562-5f51c039ef99 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:17.708241 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0639f-59be-471b-9562-5f51c039ef99 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:17.708514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0639f-59be-471b-9562-5f51c039ef99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:17.711571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e3bd0-3488-4ca6-9e66-87c249f6d4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:17.712600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e3bd0-3488-4ca6-9e66-87c249f6d4ba None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:17.713059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e3bd0-3488-4ca6-9e66-87c249f6d4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:17.713529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e3bd0-3488-4ca6-9e66-87c249f6d4ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:17.727396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e3bd0-3488-4ca6-9e66-87c249f6d4ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:17.729663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e3bd0-3488-4ca6-9e66-87c249f6d4ba None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f84e3bd0-3488-4ca6-9e66-87c249f6d4ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:17.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:17.753260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:17.754406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:17.754406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:17.754406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:17.754406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:17.760444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0639f-59be-471b-9562-5f51c039ef99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:17.760694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 365/722] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:17.762298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0639f-59be-471b-9562-5f51c039ef99 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:17.763607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e3bd0-3488-4ca6-9e66-87c249f6d4ba None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-66c0639f-59be-471b-9562-5f51c039ef99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:17.799574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:17.800706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:17.801836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:17.801836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:17.801836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:17.801836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:17.805392 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 358/723] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 567 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:17.806882 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5341a-8838-4870-bd70-531675e93db0 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:17.807171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5341a-8838-4870-bd70-531675e93db0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:17.807444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5341a-8838-4870-bd70-531675e93db0 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:17.813439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 359/724] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:17.814436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5341a-8838-4870-bd70-531675e93db0 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:17.890661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e3bd0-3488-4ca6-9e66-87c249f6d4ba None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:17.935135 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5341a-8838-4870-bd70-531675e93db0 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:17.938094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c916a8-c3af-4854-96fe-ae9daecc9402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:17.938934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c916a8-c3af-4854-96fe-ae9daecc9402 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:17.939363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c916a8-c3af-4854-96fe-ae9daecc9402 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:17.939826 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c916a8-c3af-4854-96fe-ae9daecc9402 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:17.940784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c916a8-c3af-4854-96fe-ae9daecc9402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:17.941242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c916a8-c3af-4854-96fe-ae9daecc9402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:17.943087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c916a8-c3af-4854-96fe-ae9daecc9402 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:17.954107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:17.963153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 366/725] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:17.966305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802eced-05a8-4603-a4fb-6672b3cd6b18 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:17.966776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802eced-05a8-4603-a4fb-6672b3cd6b18 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:17.967198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802eced-05a8-4603-a4fb-6672b3cd6b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:17.978895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2a17e-a64a-4d74-8f66-75035ccff3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:17.979549 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a17e-a64a-4d74-8f66-75035ccff3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:17.979919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a17e-a64a-4d74-8f66-75035ccff3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:17.980218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a17e-a64a-4d74-8f66-75035ccff3c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:17.982379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2a17e-a64a-4d74-8f66-75035ccff3c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:17.982740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2a17e-a64a-4d74-8f66-75035ccff3c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:17.984116 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2a17e-a64a-4d74-8f66-75035ccff3c4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:17.991334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8802eced-05a8-4603-a4fb-6672b3cd6b18 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:17.992916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8802eced-05a8-4603-a4fb-6672b3cd6b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:17.999976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 360/726] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:18.005543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2a17e-a64a-4d74-8f66-75035ccff3c4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:18.019262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:18.075992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 367/727] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:18.077865 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802eced-05a8-4603-a4fb-6672b3cd6b18 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:18.157619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2a17e-a64a-4d74-8f66-75035ccff3c4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:18.206448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c08483-50e5-47d9-8b29-28311300c78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:18.207383 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c08483-50e5-47d9-8b29-28311300c78e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:18.207651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802eced-05a8-4603-a4fb-6672b3cd6b18 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:18.207781 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c08483-50e5-47d9-8b29-28311300c78e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:18.207925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c08483-50e5-47d9-8b29-28311300c78e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:18.219033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c08483-50e5-47d9-8b29-28311300c78e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:18.221305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c08483-50e5-47d9-8b29-28311300c78e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:18.238874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-66c08483-50e5-47d9-8b29-28311300c78e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:18.241173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:18.243147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:18.244432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:18.244432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:18.244432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:18.244432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:18.244432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:18.244432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:18.244432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:18.244432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:18.244432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 18:04:18.244432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:18.247818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 361/728] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 18:04:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:18.248826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c08483-50e5-47d9-8b29-28311300c78e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:18.250188 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f069761-1283-4991-8e78-40fb1624bd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:18.251100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f069761-1283-4991-8e78-40fb1624bd43 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:18.251533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f069761-1283-4991-8e78-40fb1624bd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:18.251907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f069761-1283-4991-8e78-40fb1624bd43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:18.266203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f069761-1283-4991-8e78-40fb1624bd43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:18.268807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f069761-1283-4991-8e78-40fb1624bd43 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9f069761-1283-4991-8e78-40fb1624bd43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:18.288912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:18.289909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:18.290825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:18.290825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:18.290825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:18.290825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:18.290825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:18.290825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:18.290825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:18.290825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:18.296044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 368/729] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:18.297029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f069761-1283-4991-8e78-40fb1624bd43 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:18.331963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:18.368748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c08483-50e5-47d9-8b29-28311300c78e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:18.405366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc775007-23aa-4012-bad7-fcf2d24f4dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:18.406562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc775007-23aa-4012-bad7-fcf2d24f4dac None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:18.407067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc775007-23aa-4012-bad7-fcf2d24f4dac None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:18.407689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc775007-23aa-4012-bad7-fcf2d24f4dac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:18.418474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f069761-1283-4991-8e78-40fb1624bd43 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:18.419526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc775007-23aa-4012-bad7-fcf2d24f4dac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-dc775007-23aa-4012-bad7-fcf2d24f4dac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:18.427568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:18.428503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:18.429347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:18.429347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:18.429347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:18.429347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:18.429347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:18.429347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:18.429347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:18.429347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:18.429347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:18.429347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:18.429347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:04:18.429347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:18.436745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 362/730] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:04:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:18.437924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc775007-23aa-4012-bad7-fcf2d24f4dac None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:18.455424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e648c-b7fc-422b-9a73-de8b1e17177a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:18.456130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e648c-b7fc-422b-9a73-de8b1e17177a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:18.456412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e648c-b7fc-422b-9a73-de8b1e17177a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:18.456685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e648c-b7fc-422b-9a73-de8b1e17177a None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:18.457528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e648c-b7fc-422b-9a73-de8b1e17177a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:18.458034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e648c-b7fc-422b-9a73-de8b1e17177a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:18.459601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e648c-b7fc-422b-9a73-de8b1e17177a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:18.479961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 369/731] 10.222.0.46 () {56 vars in 1149 bytes} [Thu May 26 18:04:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:18.481355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e648c-b7fc-422b-9a73-de8b1e17177a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:18.533258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:18.556602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc775007-23aa-4012-bad7-fcf2d24f4dac None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:18.591017 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:18.602031 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc899af-e9e3-4131-b37b-fd9c9f3889bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:18.603072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc899af-e9e3-4131-b37b-fd9c9f3889bc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:18.603415 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e648c-b7fc-422b-9a73-de8b1e17177a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:18.603801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc899af-e9e3-4131-b37b-fd9c9f3889bc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:18.604298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc899af-e9e3-4131-b37b-fd9c9f3889bc None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:18.606472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc899af-e9e3-4131-b37b-fd9c9f3889bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:18.607019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc899af-e9e3-4131-b37b-fd9c9f3889bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:18.609067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc899af-e9e3-4131-b37b-fd9c9f3889bc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:18.625463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 363/732] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:18.627001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc899af-e9e3-4131-b37b-fd9c9f3889bc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:18.655008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fcfdf-443e-4d01-abff-5f858c37ac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:18.655748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fcfdf-443e-4d01-abff-5f858c37ac89 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:18.656106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fcfdf-443e-4d01-abff-5f858c37ac89 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:18.656346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fcfdf-443e-4d01-abff-5f858c37ac89 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:18.657104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fcfdf-443e-4d01-abff-5f858c37ac89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:18.657411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fcfdf-443e-4d01-abff-5f858c37ac89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:18.659291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fcfdf-443e-4d01-abff-5f858c37ac89 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:18.673832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 370/733] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:04:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:18.673832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fcfdf-443e-4d01-abff-5f858c37ac89 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:18.742860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc899af-e9e3-4131-b37b-fd9c9f3889bc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:18.791262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab3430-f1e7-40b3-b83d-9dd221e96659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:18.791262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab3430-f1e7-40b3-b83d-9dd221e96659 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:18.791262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab3430-f1e7-40b3-b83d-9dd221e96659 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:18.791262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab3430-f1e7-40b3-b83d-9dd221e96659 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:18.791694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fcfdf-443e-4d01-abff-5f858c37ac89 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:18.799393 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:18.832637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a283f8-e037-4a10-9dda-fb51deb46aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:18.833301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a283f8-e037-4a10-9dda-fb51deb46aae None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:18.833585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a283f8-e037-4a10-9dda-fb51deb46aae None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:18.833893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a283f8-e037-4a10-9dda-fb51deb46aae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:18.834728 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a283f8-e037-4a10-9dda-fb51deb46aae None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=d420e808730d484f80963322c5fc6856) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:18.835042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a283f8-e037-4a10-9dda-fb51deb46aae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:18.836620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a283f8-e037-4a10-9dda-fb51deb46aae None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:18.849827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 371/734] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:18 2022] GET /identity/v3/role_assignments?user.id=d420e808730d484f80963322c5fc6856&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=584beecb022b422f820e77f7623ada5e => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:18.850867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a283f8-e037-4a10-9dda-fb51deb46aae None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:18.863604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:18.882798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab3430-f1e7-40b3-b83d-9dd221e96659 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:18.885112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab3430-f1e7-40b3-b83d-9dd221e96659 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:19.093622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a283f8-e037-4a10-9dda-fb51deb46aae None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:19.099284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 364/735] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:18 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 18:04:19.100914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e85660-a35f-4e57-ba19-875a61d8a324 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:19.101395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e85660-a35f-4e57-ba19-875a61d8a324 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:19.102394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e85660-a35f-4e57-ba19-875a61d8a324 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:19.104209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 365/736] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:19.105132 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e85660-a35f-4e57-ba19-875a61d8a324 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:19.137801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11534c4-822c-4b60-99ad-1ba1ac29a19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:19.138724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11534c4-822c-4b60-99ad-1ba1ac29a19d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:19.139076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11534c4-822c-4b60-99ad-1ba1ac29a19d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:19.139417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11534c4-822c-4b60-99ad-1ba1ac29a19d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:19.155231 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11534c4-822c-4b60-99ad-1ba1ac29a19d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:19.157533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11534c4-822c-4b60-99ad-1ba1ac29a19d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a11534c4-822c-4b60-99ad-1ba1ac29a19d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:19.185310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:19.186385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:19.187289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:19.187289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:19.187289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:19.187289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:19.188078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 372/737] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:19.191037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539feffe-89c4-4608-8352-4b15ff0a4d2c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:19.191506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539feffe-89c4-4608-8352-4b15ff0a4d2c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:19.191947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539feffe-89c4-4608-8352-4b15ff0a4d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:19.216923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-539feffe-89c4-4608-8352-4b15ff0a4d2c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:19.218716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-539feffe-89c4-4608-8352-4b15ff0a4d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:19.231578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:19.247455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:19.256381 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e85660-a35f-4e57-ba19-875a61d8a324 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:19.310252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003df91-54d2-4e70-b5b3-6b564f4f7d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:19.311209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003df91-54d2-4e70-b5b3-6b564f4f7d64 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:19.311557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003df91-54d2-4e70-b5b3-6b564f4f7d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:19.312136 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003df91-54d2-4e70-b5b3-6b564f4f7d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:19.320482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 373/738] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:19.321548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539feffe-89c4-4608-8352-4b15ff0a4d2c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:19.335677 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003df91-54d2-4e70-b5b3-6b564f4f7d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:19.336974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003df91-54d2-4e70-b5b3-6b564f4f7d64 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f003df91-54d2-4e70-b5b3-6b564f4f7d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:19.376645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:19.377555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:19.378691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:19.378691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:19.378691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:19.378691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:19.378691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 366/739] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:19.379351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003df91-54d2-4e70-b5b3-6b564f4f7d64 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:19.441239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539feffe-89c4-4608-8352-4b15ff0a4d2c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:19.481359 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beaae9f-ee37-4e5c-a493-c0ba3ed01e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:19.482061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beaae9f-ee37-4e5c-a493-c0ba3ed01e54 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:19.482344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beaae9f-ee37-4e5c-a493-c0ba3ed01e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:19.482615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beaae9f-ee37-4e5c-a493-c0ba3ed01e54 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:19.483343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beaae9f-ee37-4e5c-a493-c0ba3ed01e54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:19.484181 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beaae9f-ee37-4e5c-a493-c0ba3ed01e54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:19.485903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beaae9f-ee37-4e5c-a493-c0ba3ed01e54 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:19.495472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003df91-54d2-4e70-b5b3-6b564f4f7d64 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:19.505679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 374/740] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:19.505679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beaae9f-ee37-4e5c-a493-c0ba3ed01e54 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:19.528122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:19.538981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d5d54-44b5-4b79-a265-bd3ba2b0d373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:19.539852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d5d54-44b5-4b79-a265-bd3ba2b0d373 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:19.540114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d5d54-44b5-4b79-a265-bd3ba2b0d373 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:19.540312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d5d54-44b5-4b79-a265-bd3ba2b0d373 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:19.540950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d5d54-44b5-4b79-a265-bd3ba2b0d373 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:19.542365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d5d54-44b5-4b79-a265-bd3ba2b0d373 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-d29d5d54-44b5-4b79-a265-bd3ba2b0d373 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:19.556997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:19.558695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:19.560204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 367/741] 10.222.0.46 () {56 vars in 1129 bytes} [Thu May 26 18:04:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:19.563343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a317b40-f8dc-44b6-8848-0669217d8e64 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:19.563343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a317b40-f8dc-44b6-8848-0669217d8e64 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:19.563343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a317b40-f8dc-44b6-8848-0669217d8e64 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:19.563343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 368/742] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:19.563343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a317b40-f8dc-44b6-8848-0669217d8e64 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:19.570312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:19.655019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beaae9f-ee37-4e5c-a493-c0ba3ed01e54 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:19.693000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a317b40-f8dc-44b6-8848-0669217d8e64 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:19.694114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d7118-aab6-4df1-a0ce-48165e809230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:19.694966 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d7118-aab6-4df1-a0ce-48165e809230 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:19.695285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d7118-aab6-4df1-a0ce-48165e809230 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:19.695560 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d7118-aab6-4df1-a0ce-48165e809230 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:19.696332 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d7118-aab6-4df1-a0ce-48165e809230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:19.696636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d7118-aab6-4df1-a0ce-48165e809230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:19.698063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d7118-aab6-4df1-a0ce-48165e809230 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:19.719887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 375/743] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:19.719887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d7118-aab6-4df1-a0ce-48165e809230 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:19.740291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabceae-e9d3-4c76-9b9c-ab47908d2985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:19.741131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabceae-e9d3-4c76-9b9c-ab47908d2985 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:19.741411 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabceae-e9d3-4c76-9b9c-ab47908d2985 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:19.741697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabceae-e9d3-4c76-9b9c-ab47908d2985 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:19.751377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dabceae-e9d3-4c76-9b9c-ab47908d2985 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:19.752877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dabceae-e9d3-4c76-9b9c-ab47908d2985 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-3dabceae-e9d3-4c76-9b9c-ab47908d2985 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:19.769462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:19.770488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:19.771419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:19.771419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:19.771419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:19.771419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:19.771419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:19.771419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:19.771419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:19.771419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:19.771419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 18:04:19.771419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:19.771419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 369/744] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:04:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:19.774037 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfd042-dc2b-4d19-9add-f0748d2c6a20 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:19.774329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfd042-dc2b-4d19-9add-f0748d2c6a20 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:19.774603 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfd042-dc2b-4d19-9add-f0748d2c6a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:19.794195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-3bbfd042-dc2b-4d19-9add-f0748d2c6a20 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:19.795423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3bbfd042-dc2b-4d19-9add-f0748d2c6a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:19.820573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:19.865068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d7118-aab6-4df1-a0ce-48165e809230 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:19.888679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:19.905867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 370/745] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 18:04:19.905867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfd042-dc2b-4d19-9add-f0748d2c6a20 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:19.911543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80236923-8641-41f5-946b-8e2770801c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:19.912222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80236923-8641-41f5-946b-8e2770801c03 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:19.912502 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80236923-8641-41f5-946b-8e2770801c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:19.912774 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80236923-8641-41f5-946b-8e2770801c03 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:19.914461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80236923-8641-41f5-946b-8e2770801c03 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:19.914960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80236923-8641-41f5-946b-8e2770801c03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:19.916550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80236923-8641-41f5-946b-8e2770801c03 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:19.922382 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 376/746] 10.222.0.46 () {56 vars in 1124 bytes} [Thu May 26 18:04:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:19.923456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80236923-8641-41f5-946b-8e2770801c03 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:20.035074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfd042-dc2b-4d19-9add-f0748d2c6a20 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:20.039813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80236923-8641-41f5-946b-8e2770801c03 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:20.076573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d086e-0287-4892-a81a-83a2618a730f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:20.077372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d086e-0287-4892-a81a-83a2618a730f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:20.077637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d086e-0287-4892-a81a-83a2618a730f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:20.077948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d086e-0287-4892-a81a-83a2618a730f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:20.086101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d086e-0287-4892-a81a-83a2618a730f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:20.086736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4634a2-8942-42f8-bc85-32d2bb1fc955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:20.087418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4634a2-8942-42f8-bc85-32d2bb1fc955 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:20.087587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d086e-0287-4892-a81a-83a2618a730f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:20.087827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4634a2-8942-42f8-bc85-32d2bb1fc955 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:20.088414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4634a2-8942-42f8-bc85-32d2bb1fc955 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:20.089429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4634a2-8942-42f8-bc85-32d2bb1fc955 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:20.089815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4634a2-8942-42f8-bc85-32d2bb1fc955 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:20.091708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4634a2-8942-42f8-bc85-32d2bb1fc955 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:20.095286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-0b6d086e-0287-4892-a81a-83a2618a730f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:20.102222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:20.103123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:20.104011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:20.104011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:20.104011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:20.104011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:20.104011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:20.104011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:20.104011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:20.104011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:20.104011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 371/747] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:20.104947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d086e-0287-4892-a81a-83a2618a730f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:20.116294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 377/748] 10.222.0.46 () {56 vars in 1135 bytes} [Thu May 26 18:04:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:20.117383 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4634a2-8942-42f8-bc85-32d2bb1fc955 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:20.172010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:20.226078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d086e-0287-4892-a81a-83a2618a730f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:20.229226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4634a2-8942-42f8-bc85-32d2bb1fc955 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:20.265421 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35346e6e-0cfc-420e-b415-62e77c3d1e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:20.266115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35346e6e-0cfc-420e-b415-62e77c3d1e12 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:20.266402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35346e6e-0cfc-420e-b415-62e77c3d1e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:20.266675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35346e6e-0cfc-420e-b415-62e77c3d1e12 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:20.266959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb913d-2a35-4e75-afea-0f41a7263e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:20.267393 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35346e6e-0cfc-420e-b415-62e77c3d1e12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:20.267617 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb913d-2a35-4e75-afea-0f41a7263e1f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:20.267902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb913d-2a35-4e75-afea-0f41a7263e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:20.268177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb913d-2a35-4e75-afea-0f41a7263e1f None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:20.268782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35346e6e-0cfc-420e-b415-62e77c3d1e12 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:20.268878 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb913d-2a35-4e75-afea-0f41a7263e1f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:20.269174 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb913d-2a35-4e75-afea-0f41a7263e1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:20.272520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb913d-2a35-4e75-afea-0f41a7263e1f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:20.278317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 372/749] 10.222.0.46 () {56 vars in 1124 bytes} [Thu May 26 18:04:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:20.279142 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb913d-2a35-4e75-afea-0f41a7263e1f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:20.290195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 378/750] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:04:20 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:20.291260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35346e6e-0cfc-420e-b415-62e77c3d1e12 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:20.376317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:20.446233 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb913d-2a35-4e75-afea-0f41a7263e1f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:20.447525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35346e6e-0cfc-420e-b415-62e77c3d1e12 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:20.467543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:20.491105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21384686-6968-4a0a-acea-cb501edd8fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:20.491640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c05aff-4817-4ac3-99f7-c8c1359d9a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:20.492355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21384686-6968-4a0a-acea-cb501edd8fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:20.492492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c05aff-4817-4ac3-99f7-c8c1359d9a7f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:20.492661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21384686-6968-4a0a-acea-cb501edd8fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:20.492769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c05aff-4817-4ac3-99f7-c8c1359d9a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:20.492931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c05aff-4817-4ac3-99f7-c8c1359d9a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:20.493020 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21384686-6968-4a0a-acea-cb501edd8fc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:20.493935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21384686-6968-4a0a-acea-cb501edd8fc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:20.494065 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21384686-6968-4a0a-acea-cb501edd8fc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:20.495480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21384686-6968-4a0a-acea-cb501edd8fc4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:20.516692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 373/751] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:20.517554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21384686-6968-4a0a-acea-cb501edd8fc4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:20.591446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c05aff-4817-4ac3-99f7-c8c1359d9a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:20.591446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c05aff-4817-4ac3-99f7-c8c1359d9a7f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:20.642530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21384686-6968-4a0a-acea-cb501edd8fc4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:20.653912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:20.684782 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904c9dd-9f0a-4384-ba12-1f337c8edd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:20.685460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904c9dd-9f0a-4384-ba12-1f337c8edd3b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:20.685764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904c9dd-9f0a-4384-ba12-1f337c8edd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:20.686055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904c9dd-9f0a-4384-ba12-1f337c8edd3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:20.686853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904c9dd-9f0a-4384-ba12-1f337c8edd3b None admin] RBAC: Adding query filter params (interface=public, service_id=1b82f7809cbb4a448791a2dafbbcb255, region_id=RegionOne) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:20.687310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904c9dd-9f0a-4384-ba12-1f337c8edd3b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:20.688793 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904c9dd-9f0a-4384-ba12-1f337c8edd3b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:20.702879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 374/752] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 18:04:20 2022] GET /identity/v3/endpoints?service_id=1b82f7809cbb4a448791a2dafbbcb255&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:20.703807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904c9dd-9f0a-4384-ba12-1f337c8edd3b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:20.728852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 379/753] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:04:20.730472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c05aff-4817-4ac3-99f7-c8c1359d9a7f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:20.754236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:20.821606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904c9dd-9f0a-4384-ba12-1f337c8edd3b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:20.860098 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d128fd-bea7-4923-853f-587e0e312f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:20.860917 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d128fd-bea7-4923-853f-587e0e312f5e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:20.861209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d128fd-bea7-4923-853f-587e0e312f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:20.861489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d128fd-bea7-4923-853f-587e0e312f5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:20.862372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d128fd-bea7-4923-853f-587e0e312f5e None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=90dcc2061d6b4987a1722fab33875b0a) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:20.862810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d128fd-bea7-4923-853f-587e0e312f5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:20.865438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d128fd-bea7-4923-853f-587e0e312f5e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:20.873292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 375/754] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:20 2022] GET /identity/v3/role_assignments?user.id=90dcc2061d6b4987a1722fab33875b0a&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:20.873671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c05aff-4817-4ac3-99f7-c8c1359d9a7f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:20.874180 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d128fd-bea7-4923-853f-587e0e312f5e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:20.951127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db272c-62d1-4ca5-afcd-b43e25a335ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:20.952334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db272c-62d1-4ca5-afcd-b43e25a335ae None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:20.952831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db272c-62d1-4ca5-afcd-b43e25a335ae None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:20.953386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db272c-62d1-4ca5-afcd-b43e25a335ae None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:20.954580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db272c-62d1-4ca5-afcd-b43e25a335ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:20.956316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db272c-62d1-4ca5-afcd-b43e25a335ae None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:20.966489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 380/755] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 18:04:20 2022] GET /identity/v3/services => generated 772 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:21.036484 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d128fd-bea7-4923-853f-587e0e312f5e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:21.042413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:21.116468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4a6d6-6c72-49b8-a7f7-0ecc8908d086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:21.117298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4a6d6-6c72-49b8-a7f7-0ecc8908d086 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:21.117574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4a6d6-6c72-49b8-a7f7-0ecc8908d086 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:21.117876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4a6d6-6c72-49b8-a7f7-0ecc8908d086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:21.134464 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4a6d6-6c72-49b8-a7f7-0ecc8908d086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:21.135988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4a6d6-6c72-49b8-a7f7-0ecc8908d086 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ebf4a6d6-6c72-49b8-a7f7-0ecc8908d086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:21.161161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:21.162170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:21.163076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:21.163076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:21.163076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:21.163076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:21.163076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 376/756] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:21.168808 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db272c-62d1-4ca5-afcd-b43e25a335ae None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:21.197190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be908f-5aa7-4765-9a6a-464747a9d498 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:21.197514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be908f-5aa7-4765-9a6a-464747a9d498 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:21.197768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be908f-5aa7-4765-9a6a-464747a9d498 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:21.200095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 377/757] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:21.206154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2ecbf-9fcd-475d-8cbc-b1a5626f1c6f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:21.206434 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2ecbf-9fcd-475d-8cbc-b1a5626f1c6f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:21.206627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2ecbf-9fcd-475d-8cbc-b1a5626f1c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:21.209081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:21.232700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-abf2ecbf-9fcd-475d-8cbc-b1a5626f1c6f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:21.234004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-abf2ecbf-9fcd-475d-8cbc-b1a5626f1c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:21.328764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db272c-62d1-4ca5-afcd-b43e25a335ae None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:21.341115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 378/758] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:21.365821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2ecbf-9fcd-475d-8cbc-b1a5626f1c6f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:21.379466 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461245bc-94d6-4f50-b179-6f67e23e105d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:21.380199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461245bc-94d6-4f50-b179-6f67e23e105d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:21.380474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461245bc-94d6-4f50-b179-6f67e23e105d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:21.380738 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461245bc-94d6-4f50-b179-6f67e23e105d None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:21.381514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461245bc-94d6-4f50-b179-6f67e23e105d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:21.381855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461245bc-94d6-4f50-b179-6f67e23e105d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:21.383484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461245bc-94d6-4f50-b179-6f67e23e105d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:21.400570 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 381/759] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:21.406154 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461245bc-94d6-4f50-b179-6f67e23e105d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:21.514220 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:21.522882 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:21.537893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461245bc-94d6-4f50-b179-6f67e23e105d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:21.560697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2ecbf-9fcd-475d-8cbc-b1a5626f1c6f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:21.574792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce46f5-a9a9-4b23-98d2-21f5869f29c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:21.576248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce46f5-a9a9-4b23-98d2-21f5869f29c2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:21.576741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce46f5-a9a9-4b23-98d2-21f5869f29c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:21.577230 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce46f5-a9a9-4b23-98d2-21f5869f29c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:21.584978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce46f5-a9a9-4b23-98d2-21f5869f29c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:21.586868 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce46f5-a9a9-4b23-98d2-21f5869f29c2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a8ce46f5-a9a9-4b23-98d2-21f5869f29c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:21.604783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:21.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:21.607569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:21.607569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:21.607569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:21.607569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:21.607569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:21.607569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:21.607569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:21.607569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:21.615718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 382/760] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:21.616982 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce46f5-a9a9-4b23-98d2-21f5869f29c2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:21.638889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29884b-406f-475a-92b7-2eed19650521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:21.639792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29884b-406f-475a-92b7-2eed19650521 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:21.640077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29884b-406f-475a-92b7-2eed19650521 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:21.640349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29884b-406f-475a-92b7-2eed19650521 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:21.651927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29884b-406f-475a-92b7-2eed19650521 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-9c29884b-406f-475a-92b7-2eed19650521 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:21.660216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:21.661438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:21.662290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 379/761] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:04:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:21.674879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29884b-406f-475a-92b7-2eed19650521 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:21.738683 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce46f5-a9a9-4b23-98d2-21f5869f29c2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:21.779535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc61ef-5336-4379-9d05-cb27ddc17f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:21.780635 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc61ef-5336-4379-9d05-cb27ddc17f95 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:21.781115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc61ef-5336-4379-9d05-cb27ddc17f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:21.781602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc61ef-5336-4379-9d05-cb27ddc17f95 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:21.782628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc61ef-5336-4379-9d05-cb27ddc17f95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:21.783142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc61ef-5336-4379-9d05-cb27ddc17f95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:21.785219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc61ef-5336-4379-9d05-cb27ddc17f95 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:21.787599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:21.800930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 383/762] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:21.808604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc61ef-5336-4379-9d05-cb27ddc17f95 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:21.840239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29884b-406f-475a-92b7-2eed19650521 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:21.918281 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57481007-dec2-4a71-87b0-e41fbe0f745c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:21.919476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57481007-dec2-4a71-87b0-e41fbe0f745c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:21.919770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57481007-dec2-4a71-87b0-e41fbe0f745c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:21.920039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57481007-dec2-4a71-87b0-e41fbe0f745c None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:21.921008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57481007-dec2-4a71-87b0-e41fbe0f745c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:21.921311 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57481007-dec2-4a71-87b0-e41fbe0f745c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:21.929803 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57481007-dec2-4a71-87b0-e41fbe0f745c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:21.944547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 380/763] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:04:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:21.952237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57481007-dec2-4a71-87b0-e41fbe0f745c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:22.014823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc61ef-5336-4379-9d05-cb27ddc17f95 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:22.093120 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459735ff-1e70-4fad-9c39-9554c4b909e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:22.094150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459735ff-1e70-4fad-9c39-9554c4b909e5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:22.094600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459735ff-1e70-4fad-9c39-9554c4b909e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:22.095021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459735ff-1e70-4fad-9c39-9554c4b909e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:22.096568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459735ff-1e70-4fad-9c39-9554c4b909e5 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=d656b42c2acb498caa92c77a22fd7efe) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:22.097104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459735ff-1e70-4fad-9c39-9554c4b909e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:22.099751 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459735ff-1e70-4fad-9c39-9554c4b909e5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:22.134310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 384/764] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:21 2022] GET /identity/v3/role_assignments?user.id=d656b42c2acb498caa92c77a22fd7efe&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=584beecb022b422f820e77f7623ada5e => generated 259 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:22.159563 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:22.324264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57481007-dec2-4a71-87b0-e41fbe0f745c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:22.391451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede02066-d8c9-4fd5-bb1e-569d7c11e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:22.392177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede02066-d8c9-4fd5-bb1e-569d7c11e45c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:22.392456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede02066-d8c9-4fd5-bb1e-569d7c11e45c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:22.392727 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede02066-d8c9-4fd5-bb1e-569d7c11e45c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:22.411788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede02066-d8c9-4fd5-bb1e-569d7c11e45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:22.414269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede02066-d8c9-4fd5-bb1e-569d7c11e45c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ede02066-d8c9-4fd5-bb1e-569d7c11e45c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:22.447946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:22.450722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:22.452343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:22.452343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:22.452343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:22.452343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:22.452343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 381/765] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 499 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:22.457236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459735ff-1e70-4fad-9c39-9554c4b909e5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:22.588366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:22.674024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459735ff-1e70-4fad-9c39-9554c4b909e5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:22.755180 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64a789-0e1e-41a5-9489-770dc045aa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:22.756286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64a789-0e1e-41a5-9489-770dc045aa7d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:22.756760 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64a789-0e1e-41a5-9489-770dc045aa7d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:22.757233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64a789-0e1e-41a5-9489-770dc045aa7d None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:22.758613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64a789-0e1e-41a5-9489-770dc045aa7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:22.759149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64a789-0e1e-41a5-9489-770dc045aa7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:22.762533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64a789-0e1e-41a5-9489-770dc045aa7d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:22.795855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 385/766] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:22.802851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede02066-d8c9-4fd5-bb1e-569d7c11e45c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:22.911804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:22.973371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede02066-d8c9-4fd5-bb1e-569d7c11e45c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:23.026657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f268ea-107d-461d-affc-5cb090a84553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:23.028161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f268ea-107d-461d-affc-5cb090a84553 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:23.028436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f268ea-107d-461d-affc-5cb090a84553 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:23.028711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f268ea-107d-461d-affc-5cb090a84553 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:23.047494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f268ea-107d-461d-affc-5cb090a84553 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:23.050621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f268ea-107d-461d-affc-5cb090a84553 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-32f268ea-107d-461d-affc-5cb090a84553 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:23.091715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:23.093521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:23.095735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:23.095735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:23.095735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:23.095735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:23.095735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:23.095735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:23.095735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:23.095735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:23.095735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 18:04:23.095735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:23.095735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 382/767] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 18:04:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:23.103501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64a789-0e1e-41a5-9489-770dc045aa7d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:23.281815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64a789-0e1e-41a5-9489-770dc045aa7d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:23.309293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:23.345688 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00242c73-61fb-49f9-8640-5a89854ed56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:23.346558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00242c73-61fb-49f9-8640-5a89854ed56e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:23.346892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00242c73-61fb-49f9-8640-5a89854ed56e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:23.347220 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00242c73-61fb-49f9-8640-5a89854ed56e None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:23.348127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00242c73-61fb-49f9-8640-5a89854ed56e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:23.348432 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00242c73-61fb-49f9-8640-5a89854ed56e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:23.349907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00242c73-61fb-49f9-8640-5a89854ed56e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:23.378637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 386/768] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 18:04:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:23.384494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f268ea-107d-461d-affc-5cb090a84553 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:23.486563 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccbf70-b860-475e-99ae-d43cb6c768c5 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:23.487084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccbf70-b860-475e-99ae-d43cb6c768c5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:23.487084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccbf70-b860-475e-99ae-d43cb6c768c5 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:23.488949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 387/769] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:23.505929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebb752-7193-4b31-97a3-15b1594be129 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:23.506215 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebb752-7193-4b31-97a3-15b1594be129 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:23.506493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebb752-7193-4b31-97a3-15b1594be129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:23.544705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2eebb752-7193-4b31-97a3-15b1594be129 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:23.545927 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2eebb752-7193-4b31-97a3-15b1594be129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:23.573838 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f268ea-107d-461d-affc-5cb090a84553 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:23.598520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:23.621233 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c258a1-b72a-4e09-9ac3-23ea7ec30fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:23.622766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c258a1-b72a-4e09-9ac3-23ea7ec30fcc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:23.623191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c258a1-b72a-4e09-9ac3-23ea7ec30fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:23.623589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c258a1-b72a-4e09-9ac3-23ea7ec30fcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:23.640073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c258a1-b72a-4e09-9ac3-23ea7ec30fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:23.641567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c258a1-b72a-4e09-9ac3-23ea7ec30fcc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-70c258a1-b72a-4e09-9ac3-23ea7ec30fcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:23.676138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:23.677115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:23.678021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:23.678021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:23.678021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:23.678021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:23.678021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 383/770] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:23.684454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c258a1-b72a-4e09-9ac3-23ea7ec30fcc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:23.702803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 388/771] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:23.748166 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6996d-0dd6-4ebb-ae3c-2015f439e5a1 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:23.748477 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6996d-0dd6-4ebb-ae3c-2015f439e5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:23.748716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6996d-0dd6-4ebb-ae3c-2015f439e5a1 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:23.751512 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 389/772] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:23.756062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e714c5b-cdfd-45fe-8515-a094e5129bef None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:23.756346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e714c5b-cdfd-45fe-8515-a094e5129bef None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:23.756621 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e714c5b-cdfd-45fe-8515-a094e5129bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:23.775679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8e714c5b-cdfd-45fe-8515-a094e5129bef None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:23.777959 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8e714c5b-cdfd-45fe-8515-a094e5129bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:23.866017 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c258a1-b72a-4e09-9ac3-23ea7ec30fcc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:23.869898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 390/773] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:23.870925 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e714c5b-cdfd-45fe-8515-a094e5129bef None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:23.918865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953b499-81d4-46d8-9a42-a8755a5d3a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:23.919620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953b499-81d4-46d8-9a42-a8755a5d3a24 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:23.919899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953b499-81d4-46d8-9a42-a8755a5d3a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:23.920169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953b499-81d4-46d8-9a42-a8755a5d3a24 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:23.921000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953b499-81d4-46d8-9a42-a8755a5d3a24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:23.921304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953b499-81d4-46d8-9a42-a8755a5d3a24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:23.921779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:23.924789 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953b499-81d4-46d8-9a42-a8755a5d3a24 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:23.945381 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 384/774] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:23.946702 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953b499-81d4-46d8-9a42-a8755a5d3a24 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:23.981414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:24.000624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e714c5b-cdfd-45fe-8515-a094e5129bef None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:24.046121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547a7d1-33db-44e9-9a35-a4f29f21c5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:24.047370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547a7d1-33db-44e9-9a35-a4f29f21c5cb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:24.047649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547a7d1-33db-44e9-9a35-a4f29f21c5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:24.048193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547a7d1-33db-44e9-9a35-a4f29f21c5cb None admin] PATH_INFO: `/v3/services/network` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:24.048860 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547a7d1-33db-44e9-9a35-a4f29f21c5cb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:24.050731 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547a7d1-33db-44e9-9a35-a4f29f21c5cb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:24.066634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953b499-81d4-46d8-9a42-a8755a5d3a24 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-5547a7d1-33db-44e9-9a35-a4f29f21c5cb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:24.067095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:24.068035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:24.068875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 391/775] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:23 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:24.070429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547a7d1-33db-44e9-9a35-a4f29f21c5cb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:24.114394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39df5f3-8ae2-4eed-aaf7-8f15cc0c999b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:24.115269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39df5f3-8ae2-4eed-aaf7-8f15cc0c999b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:24.115550 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39df5f3-8ae2-4eed-aaf7-8f15cc0c999b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:24.115821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39df5f3-8ae2-4eed-aaf7-8f15cc0c999b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:24.129084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39df5f3-8ae2-4eed-aaf7-8f15cc0c999b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b39df5f3-8ae2-4eed-aaf7-8f15cc0c999b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:24.138449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:24.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:24.140316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 385/776] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:24.141331 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39df5f3-8ae2-4eed-aaf7-8f15cc0c999b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:24.200742 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:24.204705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547a7d1-33db-44e9-9a35-a4f29f21c5cb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:24.246954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043a0b8-04a2-45e2-a10f-af159fb0c9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:24.247896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043a0b8-04a2-45e2-a10f-af159fb0c9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:24.248272 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043a0b8-04a2-45e2-a10f-af159fb0c9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:24.248631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043a0b8-04a2-45e2-a10f-af159fb0c9b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:24.256102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043a0b8-04a2-45e2-a10f-af159fb0c9b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:24.256617 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:24.257845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043a0b8-04a2-45e2-a10f-af159fb0c9b9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:24.266435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39df5f3-8ae2-4eed-aaf7-8f15cc0c999b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-7043a0b8-04a2-45e2-a10f-af159fb0c9b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:24.274831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:24.275805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:24.276692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:24.276692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:24.276692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:24.276692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:24.276692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:24.276692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:24.276692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:24.276692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:24.281431 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 392/777] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:24.282509 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043a0b8-04a2-45e2-a10f-af159fb0c9b9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:24.323127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5487fd1-e151-42ce-81fd-eea2727edb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:24.325122 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5487fd1-e151-42ce-81fd-eea2727edb0e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:24.325766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5487fd1-e151-42ce-81fd-eea2727edb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:24.326288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5487fd1-e151-42ce-81fd-eea2727edb0e None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:24.327474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5487fd1-e151-42ce-81fd-eea2727edb0e None admin] RBAC: Adding query filter params (name=network) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:24.328336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5487fd1-e151-42ce-81fd-eea2727edb0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:24.330467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5487fd1-e151-42ce-81fd-eea2727edb0e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:24.336867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 386/778] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:24.337936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5487fd1-e151-42ce-81fd-eea2727edb0e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:24.425476 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043a0b8-04a2-45e2-a10f-af159fb0c9b9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:24.461988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5487fd1-e151-42ce-81fd-eea2727edb0e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:24.464074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6182062-e236-4ad0-8d9f-b66afc610d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:24.465340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6182062-e236-4ad0-8d9f-b66afc610d4b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:24.465667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6182062-e236-4ad0-8d9f-b66afc610d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:24.465923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6182062-e236-4ad0-8d9f-b66afc610d4b None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:24.466902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6182062-e236-4ad0-8d9f-b66afc610d4b None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:24.467379 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6182062-e236-4ad0-8d9f-b66afc610d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:24.468937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6182062-e236-4ad0-8d9f-b66afc610d4b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:24.478484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 393/779] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:04:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:24.479624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6182062-e236-4ad0-8d9f-b66afc610d4b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:24.505196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77cab7-bd88-42df-9cfb-ab9b0071084b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:24.505892 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77cab7-bd88-42df-9cfb-ab9b0071084b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:24.506176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77cab7-bd88-42df-9cfb-ab9b0071084b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:24.506450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77cab7-bd88-42df-9cfb-ab9b0071084b None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:24.507258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77cab7-bd88-42df-9cfb-ab9b0071084b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:24.507578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77cab7-bd88-42df-9cfb-ab9b0071084b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:24.508979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77cab7-bd88-42df-9cfb-ab9b0071084b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:24.509984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:24.528208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 387/780] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:24.528208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d46763-86db-45fa-8df9-956dd56ff496 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:24.528208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d46763-86db-45fa-8df9-956dd56ff496 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:24.528208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d46763-86db-45fa-8df9-956dd56ff496 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:24.528208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 388/781] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:24.528208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d46763-86db-45fa-8df9-956dd56ff496 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:24.549521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:24.629317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6182062-e236-4ad0-8d9f-b66afc610d4b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:24.675127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d46763-86db-45fa-8df9-956dd56ff496 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:24.675380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019da73-7b14-4ab3-9e95-4a2f35a703f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:24.675948 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019da73-7b14-4ab3-9e95-4a2f35a703f1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:24.676175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019da73-7b14-4ab3-9e95-4a2f35a703f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:24.676452 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019da73-7b14-4ab3-9e95-4a2f35a703f1 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:24.677181 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019da73-7b14-4ab3-9e95-4a2f35a703f1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:24.677487 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019da73-7b14-4ab3-9e95-4a2f35a703f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:24.678949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019da73-7b14-4ab3-9e95-4a2f35a703f1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:24.690196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 394/782] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:24.690196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019da73-7b14-4ab3-9e95-4a2f35a703f1 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:24.739818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5670de5-0231-4cb6-abbb-194c7eea0b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:24.741073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5670de5-0231-4cb6-abbb-194c7eea0b2a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:24.741949 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5670de5-0231-4cb6-abbb-194c7eea0b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:24.742089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5670de5-0231-4cb6-abbb-194c7eea0b2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:24.758535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5670de5-0231-4cb6-abbb-194c7eea0b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:24.760843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5670de5-0231-4cb6-abbb-194c7eea0b2a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f5670de5-0231-4cb6-abbb-194c7eea0b2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:24.790255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:24.791221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:24.792106 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:24.792106 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:24.792106 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:24.792106 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:24.792106 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 389/783] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:24.794096 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2ac5b-50f4-4408-a096-efa7c50f0886 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:24.794587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2ac5b-50f4-4408-a096-efa7c50f0886 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:24.795078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2ac5b-50f4-4408-a096-efa7c50f0886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:24.818084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-09d2ac5b-50f4-4408-a096-efa7c50f0886 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:24.819313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-09d2ac5b-50f4-4408-a096-efa7c50f0886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:24.832200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:24.848529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019da73-7b14-4ab3-9e95-4a2f35a703f1 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:24.892838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39764332-0afa-4001-9676-6409578488fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:24.893715 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39764332-0afa-4001-9676-6409578488fa None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:24.894016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39764332-0afa-4001-9676-6409578488fa None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:24.894286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39764332-0afa-4001-9676-6409578488fa None admin] PATH_INFO: `/v3/projects/4be4045293754773bb52bbb82546a1db/users/d420e808730d484f80963322c5fc6856/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:24.896219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39764332-0afa-4001-9676-6409578488fa None admin] RBAC: Authorizing `identity:create_grant(project_id=4be4045293754773bb52bbb82546a1db, user_id=d420e808730d484f80963322c5fc6856, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:24.897661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39764332-0afa-4001-9676-6409578488fa None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:24.911409 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:24.921772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 395/784] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:04:24 2022] PUT /identity/v3/projects/4be4045293754773bb52bbb82546a1db/users/d420e808730d484f80963322c5fc6856/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:24.921772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39764332-0afa-4001-9676-6409578488fa None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:24.925649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 390/785] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:24.926727 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2ac5b-50f4-4408-a096-efa7c50f0886 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:25.084995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2ac5b-50f4-4408-a096-efa7c50f0886 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:25.099076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39764332-0afa-4001-9676-6409578488fa None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:25.130515 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19ddc4-ff3e-4089-ae0b-f842b03deffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:25.131191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19ddc4-ff3e-4089-ae0b-f842b03deffc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:25.131467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19ddc4-ff3e-4089-ae0b-f842b03deffc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:25.131735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19ddc4-ff3e-4089-ae0b-f842b03deffc None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:25.132461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19ddc4-ff3e-4089-ae0b-f842b03deffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:25.132761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19ddc4-ff3e-4089-ae0b-f842b03deffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:25.134207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19ddc4-ff3e-4089-ae0b-f842b03deffc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:25.142730 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80662b-b538-4243-b52d-25b4ea190398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:25.143459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80662b-b538-4243-b52d-25b4ea190398 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:25.143741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80662b-b538-4243-b52d-25b4ea190398 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:25.144017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80662b-b538-4243-b52d-25b4ea190398 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:25.144819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80662b-b538-4243-b52d-25b4ea190398 None admin] RBAC: Adding query filter params (interface=public, service_id=b8d4eb4696ee451cb7b833d0b9a3743b, region_id=RegionOne) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:25.145195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80662b-b538-4243-b52d-25b4ea190398 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:25.146650 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80662b-b538-4243-b52d-25b4ea190398 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:25.149436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 391/786] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:25.150326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19ddc4-ff3e-4089-ae0b-f842b03deffc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:25.161248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 396/787] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 18:04:24 2022] GET /identity/v3/endpoints?service_id=b8d4eb4696ee451cb7b833d0b9a3743b&interface=public®ion_id=RegionOne => generated 194 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:25.162532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80662b-b538-4243-b52d-25b4ea190398 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:25.177368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:25.239739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:25.286695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80662b-b538-4243-b52d-25b4ea190398 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:25.293356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19ddc4-ff3e-4089-ae0b-f842b03deffc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:25.330283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f062edd-215f-4688-9235-8643a0a9827c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:25.331151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f062edd-215f-4688-9235-8643a0a9827c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:25.331437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f062edd-215f-4688-9235-8643a0a9827c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:25.331709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f062edd-215f-4688-9235-8643a0a9827c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:25.332286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdadb61-fe99-4a56-9c5c-f431b994dba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:25.333089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdadb61-fe99-4a56-9c5c-f431b994dba7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:25.333368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdadb61-fe99-4a56-9c5c-f431b994dba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:25.333639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdadb61-fe99-4a56-9c5c-f431b994dba7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:25.334316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdadb61-fe99-4a56-9c5c-f431b994dba7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:25.335673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdadb61-fe99-4a56-9c5c-f431b994dba7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:25.341094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f062edd-215f-4688-9235-8643a0a9827c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:25.342866 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f062edd-215f-4688-9235-8643a0a9827c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-3cdadb61-fe99-4a56-9c5c-f431b994dba7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:25.347783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:25.348711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:25.349555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 392/788] 10.222.0.46 () {56 vars in 1129 bytes} [Thu May 26 18:04:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:25.350913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdadb61-fe99-4a56-9c5c-f431b994dba7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9f062edd-215f-4688-9235-8643a0a9827c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:25.364921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:25.365992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:25.367027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:25.367027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:25.367027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:25.367027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:25.367027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:25.367027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:25.367027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:25.367027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:25.367027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 18:04:25.367027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:25.368060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 397/789] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 18:04:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:25.369328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70638b-d91f-4ab4-a2ed-8446d4389c85 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:25.369652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70638b-d91f-4ab4-a2ed-8446d4389c85 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:25.370021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70638b-d91f-4ab4-a2ed-8446d4389c85 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:25.373936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 398/790] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:25.373936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70638b-d91f-4ab4-a2ed-8446d4389c85 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:25.452884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:25.465331 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdadb61-fe99-4a56-9c5c-f431b994dba7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:25.488541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:25.488723 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70638b-d91f-4ab4-a2ed-8446d4389c85 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:25.502852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598c975-e3a0-4a11-8546-0f52310780e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:25.503749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598c975-e3a0-4a11-8546-0f52310780e3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:25.504219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598c975-e3a0-4a11-8546-0f52310780e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:25.504493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598c975-e3a0-4a11-8546-0f52310780e3 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:25.505172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598c975-e3a0-4a11-8546-0f52310780e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:25.506568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598c975-e3a0-4a11-8546-0f52310780e3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:25.512018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 393/791] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 18:04:25 2022] GET /identity/v3/services => generated 772 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:25.512709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598c975-e3a0-4a11-8546-0f52310780e3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:25.539153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cfc4b-d37a-477e-ac2a-6e4fbe7318d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:25.539842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cfc4b-d37a-477e-ac2a-6e4fbe7318d5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:25.540178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cfc4b-d37a-477e-ac2a-6e4fbe7318d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:25.540464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cfc4b-d37a-477e-ac2a-6e4fbe7318d5 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:25.541189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cfc4b-d37a-477e-ac2a-6e4fbe7318d5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:25.541572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cfc4b-d37a-477e-ac2a-6e4fbe7318d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:25.543131 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cfc4b-d37a-477e-ac2a-6e4fbe7318d5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:25.549290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 399/792] 10.222.0.46 () {56 vars in 1124 bytes} [Thu May 26 18:04:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:25.550641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e1f0-2bfe-40fe-a0d6-767e2d897993 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:25.550951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e1f0-2bfe-40fe-a0d6-767e2d897993 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:25.551264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e1f0-2bfe-40fe-a0d6-767e2d897993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:25.577474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-95e0e1f0-2bfe-40fe-a0d6-767e2d897993 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:25.579119 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-95e0e1f0-2bfe-40fe-a0d6-767e2d897993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:25.645070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598c975-e3a0-4a11-8546-0f52310780e3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:25.681899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 400/793] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:25.683173 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0e1f0-2bfe-40fe-a0d6-767e2d897993 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:25.701384 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85105f22-44dc-4557-ae53-7d2ab02f0a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:25.702477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85105f22-44dc-4557-ae53-7d2ab02f0a9e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:25.703001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85105f22-44dc-4557-ae53-7d2ab02f0a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:25.703476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85105f22-44dc-4557-ae53-7d2ab02f0a9e None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:25.704510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85105f22-44dc-4557-ae53-7d2ab02f0a9e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:25.705005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85105f22-44dc-4557-ae53-7d2ab02f0a9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:25.710978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85105f22-44dc-4557-ae53-7d2ab02f0a9e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:25.736042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 394/794] 10.222.0.46 () {56 vars in 1149 bytes} [Thu May 26 18:04:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:25.737215 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85105f22-44dc-4557-ae53-7d2ab02f0a9e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:25.764017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:25.803934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:25.811334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0e1f0-2bfe-40fe-a0d6-767e2d897993 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:25.871167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cb4e0-3ccc-49a1-8235-013ba5ffc8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:25.871841 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cb4e0-3ccc-49a1-8235-013ba5ffc8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:25.872122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cb4e0-3ccc-49a1-8235-013ba5ffc8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:25.872394 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cb4e0-3ccc-49a1-8235-013ba5ffc8d8 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:25.873099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551cb4e0-3ccc-49a1-8235-013ba5ffc8d8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:25.873403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551cb4e0-3ccc-49a1-8235-013ba5ffc8d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:25.874924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551cb4e0-3ccc-49a1-8235-013ba5ffc8d8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:25.880389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 401/795] 10.222.0.46 () {56 vars in 1124 bytes} [Thu May 26 18:04:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:25.881163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cb4e0-3ccc-49a1-8235-013ba5ffc8d8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:25.891144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85105f22-44dc-4557-ae53-7d2ab02f0a9e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:25.943662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ac515-b785-439c-80eb-d2da368ccef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:25.945920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ac515-b785-439c-80eb-d2da368ccef0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:25.946724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ac515-b785-439c-80eb-d2da368ccef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:25.947191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ac515-b785-439c-80eb-d2da368ccef0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:25.963272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ac515-b785-439c-80eb-d2da368ccef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-9d7ac515-b785-439c-80eb-d2da368ccef0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:25.974712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:25.975726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:25.976735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:25.976735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:25.976735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:25.976735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:25.976735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:25.976735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:25.976735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:25.976735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:25.976735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:25.976735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:25.976735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:04:25.976735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:25.979620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 395/796] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:04:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:25.981280 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ac515-b785-439c-80eb-d2da368ccef0 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:25.996981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cb4e0-3ccc-49a1-8235-013ba5ffc8d8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:26.026312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:26.036956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f84a4e-d89f-4cca-b269-950bd12d9a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:26.037939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f84a4e-d89f-4cca-b269-950bd12d9a42 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:26.038375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f84a4e-d89f-4cca-b269-950bd12d9a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:26.038751 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f84a4e-d89f-4cca-b269-950bd12d9a42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:26.049635 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f84a4e-d89f-4cca-b269-950bd12d9a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:26.051248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f84a4e-d89f-4cca-b269-950bd12d9a42 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-82f84a4e-d89f-4cca-b269-950bd12d9a42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:26.070030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:26.071193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:26.072324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:26.072324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:26.072324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:26.072324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:26.077903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 402/797] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:26.077903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f84a4e-d89f-4cca-b269-950bd12d9a42 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:26.121501 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ac515-b785-439c-80eb-d2da368ccef0 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:26.124808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:26.162869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93322b0-1e37-46f8-89ea-9133e79720c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:26.163533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93322b0-1e37-46f8-89ea-9133e79720c7 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:26.163870 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93322b0-1e37-46f8-89ea-9133e79720c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:26.164492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93322b0-1e37-46f8-89ea-9133e79720c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:26.165555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93322b0-1e37-46f8-89ea-9133e79720c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:26.167153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93322b0-1e37-46f8-89ea-9133e79720c7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:26.197930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 396/798] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 18:04:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:26.200041 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93322b0-1e37-46f8-89ea-9133e79720c7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:26.252751 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f84a4e-d89f-4cca-b269-950bd12d9a42 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:26.351341 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107955d1-e1c3-433c-bd5d-2ca3396c1d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:26.352413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107955d1-e1c3-433c-bd5d-2ca3396c1d3f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:26.353082 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107955d1-e1c3-433c-bd5d-2ca3396c1d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:26.356137 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107955d1-e1c3-433c-bd5d-2ca3396c1d3f None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:26.357317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107955d1-e1c3-433c-bd5d-2ca3396c1d3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:26.358010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107955d1-e1c3-433c-bd5d-2ca3396c1d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:26.359861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107955d1-e1c3-433c-bd5d-2ca3396c1d3f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:26.377979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 403/799] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:04:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:26.384059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107955d1-e1c3-433c-bd5d-2ca3396c1d3f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:26.414730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93322b0-1e37-46f8-89ea-9133e79720c7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:26.458008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:26.482968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58818b-bfa1-4442-b13f-4933f887e3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:26.485025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58818b-bfa1-4442-b13f-4933f887e3fa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:26.485561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58818b-bfa1-4442-b13f-4933f887e3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:26.486430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58818b-bfa1-4442-b13f-4933f887e3fa None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:26.487654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58818b-bfa1-4442-b13f-4933f887e3fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:26.488364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58818b-bfa1-4442-b13f-4933f887e3fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:26.491384 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58818b-bfa1-4442-b13f-4933f887e3fa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:26.517404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 397/800] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:26.524043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58818b-bfa1-4442-b13f-4933f887e3fa None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:26.544199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107955d1-e1c3-433c-bd5d-2ca3396c1d3f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:26.554435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:26.588523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46213a-1e5e-4cb8-ae4f-6923c2d48cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:26.589317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46213a-1e5e-4cb8-ae4f-6923c2d48cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:26.590285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46213a-1e5e-4cb8-ae4f-6923c2d48cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:26.590564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46213a-1e5e-4cb8-ae4f-6923c2d48cc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:26.603360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46213a-1e5e-4cb8-ae4f-6923c2d48cc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:26.605863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46213a-1e5e-4cb8-ae4f-6923c2d48cc5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-1d46213a-1e5e-4cb8-ae4f-6923c2d48cc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:26.624606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:26.625554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:26.626490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:26.626490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:26.626490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:26.626490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:26.628705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 404/801] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:26.634671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46213a-1e5e-4cb8-ae4f-6923c2d48cc5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:26.655215 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58818b-bfa1-4442-b13f-4933f887e3fa None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:26.692628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c5ef2-1e1e-4adf-b109-a1ef6a0d67f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:26.693610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c5ef2-1e1e-4adf-b109-a1ef6a0d67f1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:26.693888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c5ef2-1e1e-4adf-b109-a1ef6a0d67f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:26.694165 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c5ef2-1e1e-4adf-b109-a1ef6a0d67f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:26.702565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c5ef2-1e1e-4adf-b109-a1ef6a0d67f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:26.704081 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c5ef2-1e1e-4adf-b109-a1ef6a0d67f1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-160c5ef2-1e1e-4adf-b109-a1ef6a0d67f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:26.721752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:26.722722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:26.723794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:26.723794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:26.723794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:26.723794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:26.723794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:26.723794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:26.723794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:26.723794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:26.723794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 398/802] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:26.726442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c5ef2-1e1e-4adf-b109-a1ef6a0d67f1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:26.814212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:26.830725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:26.846059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46213a-1e5e-4cb8-ae4f-6923c2d48cc5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:26.860173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c5ef2-1e1e-4adf-b109-a1ef6a0d67f1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:26.892695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cac19-3bdb-4cb3-9ea6-63b6a5d01339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:26.893364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cac19-3bdb-4cb3-9ea6-63b6a5d01339 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:26.893644 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cac19-3bdb-4cb3-9ea6-63b6a5d01339 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:26.893947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cac19-3bdb-4cb3-9ea6-63b6a5d01339 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:26.894678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cac19-3bdb-4cb3-9ea6-63b6a5d01339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:26.894980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cac19-3bdb-4cb3-9ea6-63b6a5d01339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:26.896387 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cac19-3bdb-4cb3-9ea6-63b6a5d01339 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:26.923261 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 405/803] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:26.926181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded042b1-38c7-4217-ac2d-88f2ca451daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:26.926930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded042b1-38c7-4217-ac2d-88f2ca451daf None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:26.927204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded042b1-38c7-4217-ac2d-88f2ca451daf None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:26.927478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded042b1-38c7-4217-ac2d-88f2ca451daf None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:26.928267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded042b1-38c7-4217-ac2d-88f2ca451daf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:26.928579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded042b1-38c7-4217-ac2d-88f2ca451daf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:26.930031 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded042b1-38c7-4217-ac2d-88f2ca451daf None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:26.932362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cac19-3bdb-4cb3-9ea6-63b6a5d01339 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:26.965656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 399/804] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:26.972747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded042b1-38c7-4217-ac2d-88f2ca451daf None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:27.046692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cac19-3bdb-4cb3-9ea6-63b6a5d01339 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:27.095956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248f86b-7da5-457e-946f-8b8ee7ef19dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:27.098142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248f86b-7da5-457e-946f-8b8ee7ef19dc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:27.099571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248f86b-7da5-457e-946f-8b8ee7ef19dc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:27.100043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248f86b-7da5-457e-946f-8b8ee7ef19dc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:27.117110 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248f86b-7da5-457e-946f-8b8ee7ef19dc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:27.118986 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248f86b-7da5-457e-946f-8b8ee7ef19dc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-0248f86b-7da5-457e-946f-8b8ee7ef19dc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:27.146133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:27.147142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:27.148096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:27.148096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:27.148096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:27.148096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:27.148096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:27.148096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:27.148096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:27.148096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:27.148096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 18:04:27.148096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:27.156236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 406/805] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:04:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:27.167102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248f86b-7da5-457e-946f-8b8ee7ef19dc None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:27.173250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded042b1-38c7-4217-ac2d-88f2ca451daf None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:27.193941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:27.254810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05507c40-4fe0-4c01-a967-abdb5bc233fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:27.256686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05507c40-4fe0-4c01-a967-abdb5bc233fd None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:27.257402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05507c40-4fe0-4c01-a967-abdb5bc233fd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:27.258423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05507c40-4fe0-4c01-a967-abdb5bc233fd None admin] PATH_INFO: `/v3/projects/4be4045293754773bb52bbb82546a1db/users/90dcc2061d6b4987a1722fab33875b0a/roles/d9bae602bae74c71b6c858434a426a3d` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:27.287053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:27.291625 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05507c40-4fe0-4c01-a967-abdb5bc233fd None admin] RBAC: Authorizing `identity:create_grant(project_id=4be4045293754773bb52bbb82546a1db, user_id=90dcc2061d6b4987a1722fab33875b0a, role_id=d9bae602bae74c71b6c858434a426a3d)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:27.294835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05507c40-4fe0-4c01-a967-abdb5bc233fd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:27.321278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 400/806] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:04:26 2022] PUT /identity/v3/projects/4be4045293754773bb52bbb82546a1db/users/90dcc2061d6b4987a1722fab33875b0a/roles/d9bae602bae74c71b6c858434a426a3d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:04:27.373474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248f86b-7da5-457e-946f-8b8ee7ef19dc None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:27.429304 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d099efe-8a1f-4c85-8fc1-19f54da4cc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:27.430433 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d099efe-8a1f-4c85-8fc1-19f54da4cc1e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:27.430916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d099efe-8a1f-4c85-8fc1-19f54da4cc1e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:27.431401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d099efe-8a1f-4c85-8fc1-19f54da4cc1e None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:27.432470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d099efe-8a1f-4c85-8fc1-19f54da4cc1e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:27.433014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d099efe-8a1f-4c85-8fc1-19f54da4cc1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:27.435979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d099efe-8a1f-4c85-8fc1-19f54da4cc1e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:27.462506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 407/807] 10.222.0.46 () {56 vars in 1135 bytes} [Thu May 26 18:04:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:27.471632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05507c40-4fe0-4c01-a967-abdb5bc233fd None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:27.634624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05507c40-4fe0-4c01-a967-abdb5bc233fd None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:27.683077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbe8cf-eb91-487f-8c80-3cac0106d051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:27.684443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbe8cf-eb91-487f-8c80-3cac0106d051 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:27.684718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbe8cf-eb91-487f-8c80-3cac0106d051 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:27.684978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbe8cf-eb91-487f-8c80-3cac0106d051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:27.701919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbe8cf-eb91-487f-8c80-3cac0106d051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:27.704661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbe8cf-eb91-487f-8c80-3cac0106d051 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-23cbe8cf-eb91-487f-8c80-3cac0106d051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:27.728183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:27.729926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:27.731742 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:27.731742 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:27.731742 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:27.731742 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:27.731742 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 401/808] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:27.738689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d099efe-8a1f-4c85-8fc1-19f54da4cc1e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:27.806068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e746fc-dd8e-4f88-8e91-84d70f29b270 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:27.806414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e746fc-dd8e-4f88-8e91-84d70f29b270 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:27.806820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e746fc-dd8e-4f88-8e91-84d70f29b270 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:27.810230 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 402/809] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:27.817766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dd742-d669-4bc8-808f-e289faf30747 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:27.817766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dd742-d669-4bc8-808f-e289faf30747 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:27.817766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dd742-d669-4bc8-808f-e289faf30747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:27.840325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c92dd742-d669-4bc8-808f-e289faf30747 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:27.841569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c92dd742-d669-4bc8-808f-e289faf30747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:27.914307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:27.915341 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:27.975063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 403/810] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:27.985968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d099efe-8a1f-4c85-8fc1-19f54da4cc1e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:28.033079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec4fa8-7be2-4219-afe2-9cfc7c9126ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:28.033787 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec4fa8-7be2-4219-afe2-9cfc7c9126ae None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:28.034074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec4fa8-7be2-4219-afe2-9cfc7c9126ae None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:28.034351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec4fa8-7be2-4219-afe2-9cfc7c9126ae None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:28.035092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec4fa8-7be2-4219-afe2-9cfc7c9126ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:28.035400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec4fa8-7be2-4219-afe2-9cfc7c9126ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:28.037076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec4fa8-7be2-4219-afe2-9cfc7c9126ae None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:28.044772 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dd742-d669-4bc8-808f-e289faf30747 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:28.069326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 408/811] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:28.075945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec4fa8-7be2-4219-afe2-9cfc7c9126ae None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:28.233926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dd742-d669-4bc8-808f-e289faf30747 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:28.275913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec4fa8-7be2-4219-afe2-9cfc7c9126ae None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:28.287550 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7341e1-840a-4091-b8cb-29a31a48988f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:28.288795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7341e1-840a-4091-b8cb-29a31a48988f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:28.289075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7341e1-840a-4091-b8cb-29a31a48988f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:28.289638 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7341e1-840a-4091-b8cb-29a31a48988f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:28.319432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7341e1-840a-4091-b8cb-29a31a48988f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:28.326301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-9c7341e1-840a-4091-b8cb-29a31a48988f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:28.328067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:28.328976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:28.329820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:28.329820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:28.329820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:28.329820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:28.329820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:28.329820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:28.329820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:28.329820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:28.329820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:28.329820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:28.329820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:04:28.329820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:28.333861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 404/812] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:04:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:28.335294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f34558-3d24-4bdd-9926-2c902bd812a4 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:28.335629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f34558-3d24-4bdd-9926-2c902bd812a4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:28.335993 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f34558-3d24-4bdd-9926-2c902bd812a4 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:28.337530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 405/813] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:28.341914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f34558-3d24-4bdd-9926-2c902bd812a4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:28.346423 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d13585-537e-44c8-84fa-8db1cf5505a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:28.347516 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d13585-537e-44c8-84fa-8db1cf5505a7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:28.348017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d13585-537e-44c8-84fa-8db1cf5505a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:28.348513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d13585-537e-44c8-84fa-8db1cf5505a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:28.358824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d13585-537e-44c8-84fa-8db1cf5505a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:28.361162 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d13585-537e-44c8-84fa-8db1cf5505a7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:28.375369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-47d13585-537e-44c8-84fa-8db1cf5505a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:28.383526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:28.384457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:28.385546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:28.385546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:28.385546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:28.385546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:28.385546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:28.385546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:28.385546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:28.385546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:28.390127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 409/814] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:28.392137 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5960b5-2a9b-4e5b-b65f-f7bc29ab209c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:28.392922 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5960b5-2a9b-4e5b-b65f-f7bc29ab209c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:28.393414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5960b5-2a9b-4e5b-b65f-f7bc29ab209c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:28.415388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8e5960b5-2a9b-4e5b-b65f-f7bc29ab209c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:28.417002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8e5960b5-2a9b-4e5b-b65f-f7bc29ab209c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:28.464386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f34558-3d24-4bdd-9926-2c902bd812a4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:28.500865 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 410/815] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:28.502327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5960b5-2a9b-4e5b-b65f-f7bc29ab209c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:28.502981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370a70d-81dc-4de5-8440-30cda00ea244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:28.503657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370a70d-81dc-4de5-8440-30cda00ea244 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:28.503938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370a70d-81dc-4de5-8440-30cda00ea244 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:28.504211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370a70d-81dc-4de5-8440-30cda00ea244 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:28.504960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370a70d-81dc-4de5-8440-30cda00ea244 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:28.505267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370a70d-81dc-4de5-8440-30cda00ea244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:28.513754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370a70d-81dc-4de5-8440-30cda00ea244 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:28.521860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 406/816] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:04:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:28.523990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370a70d-81dc-4de5-8440-30cda00ea244 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:28.628347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:28.637917 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370a70d-81dc-4de5-8440-30cda00ea244 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:28.663897 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5960b5-2a9b-4e5b-b65f-f7bc29ab209c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:28.676470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b232863-6935-4539-a556-526af4a7347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:28.677616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b232863-6935-4539-a556-526af4a7347c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:28.678137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b232863-6935-4539-a556-526af4a7347c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:28.678601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b232863-6935-4539-a556-526af4a7347c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:28.685713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:28.692098 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b232863-6935-4539-a556-526af4a7347c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:28.694594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b232863-6935-4539-a556-526af4a7347c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:28.708093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adb53b-d0dc-4af7-bb6c-6e906ede0b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:28.708811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adb53b-d0dc-4af7-bb6c-6e906ede0b38 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:28.709095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adb53b-d0dc-4af7-bb6c-6e906ede0b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:28.709367 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adb53b-d0dc-4af7-bb6c-6e906ede0b38 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:28.710208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73adb53b-d0dc-4af7-bb6c-6e906ede0b38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:28.710519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73adb53b-d0dc-4af7-bb6c-6e906ede0b38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:28.712043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73adb53b-d0dc-4af7-bb6c-6e906ede0b38 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-5b232863-6935-4539-a556-526af4a7347c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:28.717052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:28.717987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:28.718874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:28.718874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:28.718874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:28.718874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:28.721110 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 407/817] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:28.722204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b232863-6935-4539-a556-526af4a7347c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:28.732131 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 411/818] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:28.732941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adb53b-d0dc-4af7-bb6c-6e906ede0b38 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:28.833291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b232863-6935-4539-a556-526af4a7347c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:28.853617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adb53b-d0dc-4af7-bb6c-6e906ede0b38 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:28.880792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941f72e-cb44-4984-b51c-319f578cb747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:28.880792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941f72e-cb44-4984-b51c-319f578cb747 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:28.880792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941f72e-cb44-4984-b51c-319f578cb747 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:28.880792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941f72e-cb44-4984-b51c-319f578cb747 None admin] PATH_INFO: `/v3/services/network` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:28.881262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941f72e-cb44-4984-b51c-319f578cb747 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:28.881262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941f72e-cb44-4984-b51c-319f578cb747 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:28.892695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860b2b8-1d87-4c3e-a8e1-e493cbc77bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:28.893363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860b2b8-1d87-4c3e-a8e1-e493cbc77bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:28.893644 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860b2b8-1d87-4c3e-a8e1-e493cbc77bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:28.893983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860b2b8-1d87-4c3e-a8e1-e493cbc77bd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:28.894798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860b2b8-1d87-4c3e-a8e1-e493cbc77bd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:28.895106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860b2b8-1d87-4c3e-a8e1-e493cbc77bd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f941f72e-cb44-4984-b51c-319f578cb747 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:28.895670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:28.897548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 18:04:28.898602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:28.899881 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860b2b8-1d87-4c3e-a8e1-e493cbc77bd5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:28.900030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 408/819] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:28 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:28.900030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941f72e-cb44-4984-b51c-319f578cb747 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:28.912101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 412/820] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:28.912899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860b2b8-1d87-4c3e-a8e1-e493cbc77bd5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:28.930168 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:28.990147 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:29.043694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941f72e-cb44-4984-b51c-319f578cb747 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:29.090337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c54463-a047-4506-b8e6-9ebd69b1e35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:29.091615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c54463-a047-4506-b8e6-9ebd69b1e35c None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:29.092095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c54463-a047-4506-b8e6-9ebd69b1e35c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:29.092539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c54463-a047-4506-b8e6-9ebd69b1e35c None admin] PATH_INFO: `/v3/projects/4be4045293754773bb52bbb82546a1db/users/d656b42c2acb498caa92c77a22fd7efe/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:29.095466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c54463-a047-4506-b8e6-9ebd69b1e35c None admin] RBAC: Authorizing `identity:create_grant(project_id=4be4045293754773bb52bbb82546a1db, user_id=d656b42c2acb498caa92c77a22fd7efe, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:29.097373 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c54463-a047-4506-b8e6-9ebd69b1e35c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:29.099535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860b2b8-1d87-4c3e-a8e1-e493cbc77bd5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:29.113748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 409/821] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:04:28 2022] PUT /identity/v3/projects/4be4045293754773bb52bbb82546a1db/users/d656b42c2acb498caa92c77a22fd7efe/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:29.114998 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c54463-a047-4506-b8e6-9ebd69b1e35c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:29.190032 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399c602-3527-4f27-8b6a-a0da8f71c50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:29.191033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399c602-3527-4f27-8b6a-a0da8f71c50f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:29.191488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399c602-3527-4f27-8b6a-a0da8f71c50f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:29.192015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399c602-3527-4f27-8b6a-a0da8f71c50f None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:29.194090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399c602-3527-4f27-8b6a-a0da8f71c50f None admin] RBAC: Adding query filter params (name=network) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:29.194627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399c602-3527-4f27-8b6a-a0da8f71c50f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:29.197264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399c602-3527-4f27-8b6a-a0da8f71c50f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:29.209300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 413/822] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:29.217138 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399c602-3527-4f27-8b6a-a0da8f71c50f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:29.270012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:29.329225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c54463-a047-4506-b8e6-9ebd69b1e35c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:29.383917 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060569c3-1914-4758-bdfd-6234895a456a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:29.385366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060569c3-1914-4758-bdfd-6234895a456a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:29.385893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060569c3-1914-4758-bdfd-6234895a456a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:29.387093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060569c3-1914-4758-bdfd-6234895a456a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:29.399315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060569c3-1914-4758-bdfd-6234895a456a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:29.400769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060569c3-1914-4758-bdfd-6234895a456a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:29.418548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399c602-3527-4f27-8b6a-a0da8f71c50f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:29.420223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-060569c3-1914-4758-bdfd-6234895a456a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:29.427163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:29.428153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:29.429104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:29.429104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:29.429104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:29.429104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:29.429104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:29.429104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:29.429104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:29.429104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:29.429104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 18:04:29.429104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:29.431881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 410/823] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 18:04:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:29.433477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db64951-b1ac-4fc7-a770-06af3adf23d9 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:29.434169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db64951-b1ac-4fc7-a770-06af3adf23d9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:29.434733 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db64951-b1ac-4fc7-a770-06af3adf23d9 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:29.436712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 411/824] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:29.437657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db64951-b1ac-4fc7-a770-06af3adf23d9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:29.457364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62651c-7517-4773-830c-a22ab6ce0f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:29.458168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62651c-7517-4773-830c-a22ab6ce0f4b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:29.458343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62651c-7517-4773-830c-a22ab6ce0f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:29.458610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62651c-7517-4773-830c-a22ab6ce0f4b None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:29.459320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62651c-7517-4773-830c-a22ab6ce0f4b None admin] RBAC: Adding query filter params (type=network) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:29.459625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62651c-7517-4773-830c-a22ab6ce0f4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:29.461149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62651c-7517-4773-830c-a22ab6ce0f4b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:29.467921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 414/825] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:29.469301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e7149-9d38-4dfc-83bb-ab15ded7ccdf None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:29.469591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e7149-9d38-4dfc-83bb-ab15ded7ccdf None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:29.469913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e7149-9d38-4dfc-83bb-ab15ded7ccdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:29.488702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-068e7149-9d38-4dfc-83bb-ab15ded7ccdf None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:29.490078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-068e7149-9d38-4dfc-83bb-ab15ded7ccdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:29.577401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 415/826] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:29.577401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e7149-9d38-4dfc-83bb-ab15ded7ccdf None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:29.587447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db64951-b1ac-4fc7-a770-06af3adf23d9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:29.633833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:29.641392 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bde56-88c0-4e21-9c47-9561781c1f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:29.642450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bde56-88c0-4e21-9c47-9561781c1f06 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:29.642995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bde56-88c0-4e21-9c47-9561781c1f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:29.643448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bde56-88c0-4e21-9c47-9561781c1f06 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:29.644456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bde56-88c0-4e21-9c47-9561781c1f06 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:29.644968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bde56-88c0-4e21-9c47-9561781c1f06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:29.646696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bde56-88c0-4e21-9c47-9561781c1f06 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:29.667810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 412/827] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 18:04:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:29.668962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bde56-88c0-4e21-9c47-9561781c1f06 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:29.692663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:29.698903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e7149-9d38-4dfc-83bb-ab15ded7ccdf None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:29.738379 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b961e00-3de5-4dc0-a305-fa6df3618cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:29.739510 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b961e00-3de5-4dc0-a305-fa6df3618cf5 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:29.739798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b961e00-3de5-4dc0-a305-fa6df3618cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:29.740073 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b961e00-3de5-4dc0-a305-fa6df3618cf5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:29.741102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b961e00-3de5-4dc0-a305-fa6df3618cf5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:29.742603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b961e00-3de5-4dc0-a305-fa6df3618cf5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:29.774255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 416/828] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 18:04:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:29.775602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b961e00-3de5-4dc0-a305-fa6df3618cf5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:29.788012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bde56-88c0-4e21-9c47-9561781c1f06 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:29.906579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e504f8-6df7-409f-890a-bd272b9666dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:29.907771 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e504f8-6df7-409f-890a-bd272b9666dd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:29.908236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e504f8-6df7-409f-890a-bd272b9666dd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:29.908684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e504f8-6df7-409f-890a-bd272b9666dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:29.909551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e504f8-6df7-409f-890a-bd272b9666dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:29.911477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e504f8-6df7-409f-890a-bd272b9666dd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:29.918730 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b961e00-3de5-4dc0-a305-fa6df3618cf5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a2e504f8-6df7-409f-890a-bd272b9666dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:29.942884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:29.943958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 18:04:29.944802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:29.947488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 413/829] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:04:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:29.954031 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e504f8-6df7-409f-890a-bd272b9666dd None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:29.966547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541bf2c-a8b9-4dda-8595-cc5db582f89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:29.967217 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541bf2c-a8b9-4dda-8595-cc5db582f89b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:29.967501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541bf2c-a8b9-4dda-8595-cc5db582f89b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:29.967780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541bf2c-a8b9-4dda-8595-cc5db582f89b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:30.001695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:30.054233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541bf2c-a8b9-4dda-8595-cc5db582f89b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:30.055819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541bf2c-a8b9-4dda-8595-cc5db582f89b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:30.062799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:30.126274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e504f8-6df7-409f-890a-bd272b9666dd None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:30.167734 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 417/830] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:04:30.169569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9530d43-30bb-40f4-9435-43ef47645633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:30.170027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5bd34-6796-4cc9-b415-73d90403579d None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:30.170513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5bd34-6796-4cc9-b415-73d90403579d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:30.170647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9530d43-30bb-40f4-9435-43ef47645633 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:30.171075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9530d43-30bb-40f4-9435-43ef47645633 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:30.171255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5bd34-6796-4cc9-b415-73d90403579d None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:30.171494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9530d43-30bb-40f4-9435-43ef47645633 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:30.172966 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9530d43-30bb-40f4-9435-43ef47645633 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:30.173419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 418/831] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:30.173532 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9530d43-30bb-40f4-9435-43ef47645633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:30.174432 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5bd34-6796-4cc9-b415-73d90403579d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:30.177226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9530d43-30bb-40f4-9435-43ef47645633 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:30.184927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 414/832] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:04:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:30.186974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d54626-900f-481a-a692-0acd67024465 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:30.187400 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d54626-900f-481a-a692-0acd67024465 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:30.187807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d54626-900f-481a-a692-0acd67024465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:30.219754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-59d54626-900f-481a-a692-0acd67024465 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:30.220964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-59d54626-900f-481a-a692-0acd67024465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:30.243028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:30.294926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 415/833] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:30.296057 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d54626-900f-481a-a692-0acd67024465 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:30.306423 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5bd34-6796-4cc9-b415-73d90403579d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:30.349187 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:30.349384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea5b0d-65f5-419a-a283-4a76c78e2988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:30.350248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea5b0d-65f5-419a-a283-4a76c78e2988 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:30.350670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea5b0d-65f5-419a-a283-4a76c78e2988 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:30.350999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea5b0d-65f5-419a-a283-4a76c78e2988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:30.362229 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea5b0d-65f5-419a-a283-4a76c78e2988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:30.363769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea5b0d-65f5-419a-a283-4a76c78e2988 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f9ea5b0d-65f5-419a-a283-4a76c78e2988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:30.381991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:30.382943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:30.383819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:30.383819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:30.383819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:30.383819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:30.383819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 419/834] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:30.384547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea5b0d-65f5-419a-a283-4a76c78e2988 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:30.419480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d54626-900f-481a-a692-0acd67024465 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:30.455538 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572380bf-6757-4cd4-9e79-51e5168ed950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:30.456200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572380bf-6757-4cd4-9e79-51e5168ed950 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:30.456478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572380bf-6757-4cd4-9e79-51e5168ed950 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:30.456748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572380bf-6757-4cd4-9e79-51e5168ed950 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:30.457444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572380bf-6757-4cd4-9e79-51e5168ed950 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:30.457772 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572380bf-6757-4cd4-9e79-51e5168ed950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:30.459133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572380bf-6757-4cd4-9e79-51e5168ed950 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:30.469780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 416/835] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:04:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:30.469780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572380bf-6757-4cd4-9e79-51e5168ed950 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:30.520250 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea5b0d-65f5-419a-a283-4a76c78e2988 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:30.526636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:30.567520 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c1474-adcc-412e-8e7a-9f8e7356ccbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:30.568546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c1474-adcc-412e-8e7a-9f8e7356ccbd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:30.568932 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c1474-adcc-412e-8e7a-9f8e7356ccbd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:30.569302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c1474-adcc-412e-8e7a-9f8e7356ccbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:30.586901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c1474-adcc-412e-8e7a-9f8e7356ccbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:30.598135 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572380bf-6757-4cd4-9e79-51e5168ed950 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-4d8c1474-adcc-412e-8e7a-9f8e7356ccbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:30.598794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:30.599720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:30.600555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:30.600555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:30.600555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:30.600555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:30.600555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:30.600555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:30.600555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:30.600555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:30.600555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:30.600555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:30.600555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:04:30.600555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:30.602000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 420/836] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:04:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:30.607507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c1474-adcc-412e-8e7a-9f8e7356ccbd None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:30.612026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:30.644624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92268132-1e4a-4987-a9f0-a8d52e66e497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:30.645294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92268132-1e4a-4987-a9f0-a8d52e66e497 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:30.645575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92268132-1e4a-4987-a9f0-a8d52e66e497 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:30.645880 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92268132-1e4a-4987-a9f0-a8d52e66e497 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:30.646610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92268132-1e4a-4987-a9f0-a8d52e66e497 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:30.647001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92268132-1e4a-4987-a9f0-a8d52e66e497 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:30.648287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92268132-1e4a-4987-a9f0-a8d52e66e497 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:30.663836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 417/837] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:30.671014 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92268132-1e4a-4987-a9f0-a8d52e66e497 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:30.741146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c1474-adcc-412e-8e7a-9f8e7356ccbd None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:30.779559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d174c-10dd-4209-8b4a-e93fd4cbfcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:30.780473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d174c-10dd-4209-8b4a-e93fd4cbfcad None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:30.780927 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d174c-10dd-4209-8b4a-e93fd4cbfcad None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:30.781356 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d174c-10dd-4209-8b4a-e93fd4cbfcad None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:30.782403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d174c-10dd-4209-8b4a-e93fd4cbfcad None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:30.782999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d174c-10dd-4209-8b4a-e93fd4cbfcad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:30.784599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d174c-10dd-4209-8b4a-e93fd4cbfcad None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:30.785315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92268132-1e4a-4987-a9f0-a8d52e66e497 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:30.798572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 421/838] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:04:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:30.799939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d174c-10dd-4209-8b4a-e93fd4cbfcad None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:30.821802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eab67-40c3-4db7-9b6d-f21c78238473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:30.822593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eab67-40c3-4db7-9b6d-f21c78238473 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:30.822869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eab67-40c3-4db7-9b6d-f21c78238473 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:30.823137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eab67-40c3-4db7-9b6d-f21c78238473 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:30.830879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9eab67-40c3-4db7-9b6d-f21c78238473 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:30.832316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9eab67-40c3-4db7-9b6d-f21c78238473 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:30.835030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ed9eab67-40c3-4db7-9b6d-f21c78238473 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:30.847848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:30.848802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:30.849695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:30.849695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:30.849695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:30.849695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:30.849695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:30.849695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:30.849695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:30.849695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:30.853791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 418/839] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:30.855061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eab67-40c3-4db7-9b6d-f21c78238473 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:30.886753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:30.931079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d174c-10dd-4209-8b4a-e93fd4cbfcad None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:30.965690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eab67-40c3-4db7-9b6d-f21c78238473 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:30.967449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f29a7-bc91-4f6b-b8d3-2f1e0a55c358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:30.968529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f29a7-bc91-4f6b-b8d3-2f1e0a55c358 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:30.969051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f29a7-bc91-4f6b-b8d3-2f1e0a55c358 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:30.969534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f29a7-bc91-4f6b-b8d3-2f1e0a55c358 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:30.982910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f29a7-bc91-4f6b-b8d3-2f1e0a55c358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:30.984834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f29a7-bc91-4f6b-b8d3-2f1e0a55c358 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f54f29a7-bc91-4f6b-b8d3-2f1e0a55c358 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:31.003599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:31.004518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:31.005413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:31.005413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:31.005413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:31.005413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:31.005626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee59fb-0576-44c6-81b2-fa3dee941329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:31.006180 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee59fb-0576-44c6-81b2-fa3dee941329 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:31.006362 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee59fb-0576-44c6-81b2-fa3dee941329 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:31.006570 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee59fb-0576-44c6-81b2-fa3dee941329 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:31.007414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee59fb-0576-44c6-81b2-fa3dee941329 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:31.007721 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee59fb-0576-44c6-81b2-fa3dee941329 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:31.008746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 422/840] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:31.009338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee59fb-0576-44c6-81b2-fa3dee941329 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:31.014088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f29a7-bc91-4f6b-b8d3-2f1e0a55c358 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:31.027720 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 419/841] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:31.029140 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee59fb-0576-44c6-81b2-fa3dee941329 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:31.098920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:31.122013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f29a7-bc91-4f6b-b8d3-2f1e0a55c358 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:31.124154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:31.142155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee59fb-0576-44c6-81b2-fa3dee941329 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:31.157122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840dde59-96ed-4278-999b-a332d2aaa4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:31.157810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dde59-96ed-4278-999b-a332d2aaa4db None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:31.158085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dde59-96ed-4278-999b-a332d2aaa4db None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:31.158353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dde59-96ed-4278-999b-a332d2aaa4db None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:31.159068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840dde59-96ed-4278-999b-a332d2aaa4db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:31.159370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840dde59-96ed-4278-999b-a332d2aaa4db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:31.160767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840dde59-96ed-4278-999b-a332d2aaa4db None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:31.177974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 423/842] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:31.179761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840dde59-96ed-4278-999b-a332d2aaa4db None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:31.189413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d268ccb-cb67-484e-9c7a-ec219701f512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:31.190040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d268ccb-cb67-484e-9c7a-ec219701f512 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:31.190315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d268ccb-cb67-484e-9c7a-ec219701f512 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:31.190586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d268ccb-cb67-484e-9c7a-ec219701f512 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:31.191395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d268ccb-cb67-484e-9c7a-ec219701f512 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=d420e808730d484f80963322c5fc6856) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:31.191713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d268ccb-cb67-484e-9c7a-ec219701f512 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:31.193527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d268ccb-cb67-484e-9c7a-ec219701f512 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:31.202197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 420/843] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:31 2022] GET /identity/v3/role_assignments?user.id=d420e808730d484f80963322c5fc6856&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=584beecb022b422f820e77f7623ada5e => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:31.305797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840dde59-96ed-4278-999b-a332d2aaa4db None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:31.347233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda675d4-afe8-403b-90a5-c6abaf73b6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:31.348073 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda675d4-afe8-403b-90a5-c6abaf73b6bb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:31.348353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda675d4-afe8-403b-90a5-c6abaf73b6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:31.348625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda675d4-afe8-403b-90a5-c6abaf73b6bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:31.358302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda675d4-afe8-403b-90a5-c6abaf73b6bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:31.359866 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda675d4-afe8-403b-90a5-c6abaf73b6bb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:31.362125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-fda675d4-afe8-403b-90a5-c6abaf73b6bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:31.380761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:31.381817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:31.383026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:31.383026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:31.383026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:31.383026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:31.383026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:31.383026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:31.383026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:31.383026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:31.383026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 18:04:31.383026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:31.383026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 424/844] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 18:04:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:31.388250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d268ccb-cb67-484e-9c7a-ec219701f512 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:31.497994 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d268ccb-cb67-484e-9c7a-ec219701f512 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:31.533785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41d8e8-9140-476e-8cbb-a3bf447dd2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:31.534453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41d8e8-9140-476e-8cbb-a3bf447dd2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:31.534732 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41d8e8-9140-476e-8cbb-a3bf447dd2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:31.535004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41d8e8-9140-476e-8cbb-a3bf447dd2a6 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:31.535833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41d8e8-9140-476e-8cbb-a3bf447dd2a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:31.536584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41d8e8-9140-476e-8cbb-a3bf447dd2a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:31.538036 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41d8e8-9140-476e-8cbb-a3bf447dd2a6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:31.550573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd943c-447b-47f0-b10c-0bae20a79fe3 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:31.551139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd943c-447b-47f0-b10c-0bae20a79fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:31.551593 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd943c-447b-47f0-b10c-0bae20a79fe3 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:31.553620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 425/845] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:31.559895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 421/846] 10.222.0.46 () {56 vars in 1149 bytes} [Thu May 26 18:04:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:31.561499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888bfd7-008c-4e59-8a53-c4854bccda1b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:31.561936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888bfd7-008c-4e59-8a53-c4854bccda1b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:31.562318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888bfd7-008c-4e59-8a53-c4854bccda1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:31.565343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41d8e8-9140-476e-8cbb-a3bf447dd2a6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:31.584938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:31.592491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2888bfd7-008c-4e59-8a53-c4854bccda1b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:31.594073 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2888bfd7-008c-4e59-8a53-c4854bccda1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:31.676528 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41d8e8-9140-476e-8cbb-a3bf447dd2a6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:31.685814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 426/847] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:31.714665 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888bfd7-008c-4e59-8a53-c4854bccda1b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:31.720268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531669f8-2cf4-414d-931d-4f58aa92d5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:31.720928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531669f8-2cf4-414d-931d-4f58aa92d5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:31.721208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531669f8-2cf4-414d-931d-4f58aa92d5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:31.721477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531669f8-2cf4-414d-931d-4f58aa92d5e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:31.809062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531669f8-2cf4-414d-931d-4f58aa92d5e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:31.810908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531669f8-2cf4-414d-931d-4f58aa92d5e0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:31.825304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:31.844705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888bfd7-008c-4e59-8a53-c4854bccda1b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:31.881983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:31.891672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14a299-c7a8-4e22-bbec-f5487560a213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:31.892643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14a299-c7a8-4e22-bbec-f5487560a213 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:31.893029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14a299-c7a8-4e22-bbec-f5487560a213 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:31.893408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14a299-c7a8-4e22-bbec-f5487560a213 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:31.909397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14a299-c7a8-4e22-bbec-f5487560a213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8a14a299-c7a8-4e22-bbec-f5487560a213 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:31.920047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:31.921061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:31.921926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:31.921926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:31.921926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:31.921926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:31.921926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:31.921926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:31.921926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:31.921926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:31.921926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:31.921926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:31.921926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:04:31.921926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:31.922576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 427/848] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:04:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:31.927508 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14a299-c7a8-4e22-bbec-f5487560a213 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:31.956456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 422/849] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:31 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:04:31.963862 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531669f8-2cf4-414d-931d-4f58aa92d5e0 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:32.067171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14a299-c7a8-4e22-bbec-f5487560a213 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:32.107565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531669f8-2cf4-414d-931d-4f58aa92d5e0 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:32.108414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bfd90-727e-485d-8311-0acd43350ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:32.109311 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bfd90-727e-485d-8311-0acd43350ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:32.109699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bfd90-727e-485d-8311-0acd43350ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:32.110117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bfd90-727e-485d-8311-0acd43350ed8 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:32.110994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bfd90-727e-485d-8311-0acd43350ed8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:32.111375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bfd90-727e-485d-8311-0acd43350ed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:32.112860 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bfd90-727e-485d-8311-0acd43350ed8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:32.120628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:32.123268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 428/850] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:04:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:32.128983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bfd90-727e-485d-8311-0acd43350ed8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:32.158647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec155b-8c60-49e6-b6ef-9c8d097f03cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:32.159544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec155b-8c60-49e6-b6ef-9c8d097f03cf None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:32.159887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec155b-8c60-49e6-b6ef-9c8d097f03cf None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:32.160459 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec155b-8c60-49e6-b6ef-9c8d097f03cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:32.172735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec155b-8c60-49e6-b6ef-9c8d097f03cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:32.174479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec155b-8c60-49e6-b6ef-9c8d097f03cf None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-59ec155b-8c60-49e6-b6ef-9c8d097f03cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:32.193633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:32.194604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:32.195500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:32.195500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:32.195500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:32.195500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:32.200816 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 423/851] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:32.201522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec155b-8c60-49e6-b6ef-9c8d097f03cf None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:32.206541 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:32.270138 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bfd90-727e-485d-8311-0acd43350ed8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:32.322832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62716d44-b6f7-499a-97ef-073ddc25288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:32.323860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec155b-8c60-49e6-b6ef-9c8d097f03cf None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:32.324014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62716d44-b6f7-499a-97ef-073ddc25288a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:32.324430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62716d44-b6f7-499a-97ef-073ddc25288a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:32.324843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62716d44-b6f7-499a-97ef-073ddc25288a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:32.337109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62716d44-b6f7-499a-97ef-073ddc25288a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:32.339034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62716d44-b6f7-499a-97ef-073ddc25288a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-62716d44-b6f7-499a-97ef-073ddc25288a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:32.359612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:32.360737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:32.361863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:32.361863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:32.361863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:32.361863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:32.362927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded1421-2f52-4dc5-9c32-1be058464495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:32.365801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded1421-2f52-4dc5-9c32-1be058464495 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:32.365801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded1421-2f52-4dc5-9c32-1be058464495 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:32.365801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded1421-2f52-4dc5-9c32-1be058464495 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:32.365801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded1421-2f52-4dc5-9c32-1be058464495 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:32.365801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded1421-2f52-4dc5-9c32-1be058464495 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:32.367116 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded1421-2f52-4dc5-9c32-1be058464495 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:32.368890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 429/852] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:32.369778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62716d44-b6f7-499a-97ef-073ddc25288a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:32.380346 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 424/853] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:32.387585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded1421-2f52-4dc5-9c32-1be058464495 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:32.413489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:32.488006 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62716d44-b6f7-499a-97ef-073ddc25288a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:32.489993 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:32.507188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded1421-2f52-4dc5-9c32-1be058464495 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:32.528933 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f92e7-9374-462f-9e87-5d6470050914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:32.529930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f92e7-9374-462f-9e87-5d6470050914 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:32.530392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f92e7-9374-462f-9e87-5d6470050914 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:32.530809 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f92e7-9374-462f-9e87-5d6470050914 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:32.531717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f92e7-9374-462f-9e87-5d6470050914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:32.532185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f92e7-9374-462f-9e87-5d6470050914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:32.533814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f92e7-9374-462f-9e87-5d6470050914 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:32.551939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed5fa-81d9-4e48-b97d-729e70f7ab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:32.551939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed5fa-81d9-4e48-b97d-729e70f7ab96 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:32.551939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed5fa-81d9-4e48-b97d-729e70f7ab96 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:32.551939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed5fa-81d9-4e48-b97d-729e70f7ab96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:32.552678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 430/854] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:32.553741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f92e7-9374-462f-9e87-5d6470050914 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:32.559801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ed5fa-81d9-4e48-b97d-729e70f7ab96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:32.561427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ed5fa-81d9-4e48-b97d-729e70f7ab96 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-6d2ed5fa-81d9-4e48-b97d-729e70f7ab96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:32.582758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:32.583726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:32.584614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:32.584614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:32.584614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:32.584614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:32.584614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:32.584614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:32.584614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:32.584614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:32.588983 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 425/855] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:32.590704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed5fa-81d9-4e48-b97d-729e70f7ab96 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:32.687743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f92e7-9374-462f-9e87-5d6470050914 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:32.695014 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:32.721670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed5fa-81d9-4e48-b97d-729e70f7ab96 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:32.740775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586a273-cc08-4228-a158-461a19d5a554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:32.741617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a273-cc08-4228-a158-461a19d5a554 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:32.742109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a273-cc08-4228-a158-461a19d5a554 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:32.742393 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a273-cc08-4228-a158-461a19d5a554 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:32.753806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586a273-cc08-4228-a158-461a19d5a554 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:32.756185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586a273-cc08-4228-a158-461a19d5a554 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:32.769613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f2aab-fc85-41bd-b7ea-35531ef1adf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:32.770315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f2aab-fc85-41bd-b7ea-35531ef1adf4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:32.770594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f2aab-fc85-41bd-b7ea-35531ef1adf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:32.770867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f2aab-fc85-41bd-b7ea-35531ef1adf4 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:32.771655 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f2aab-fc85-41bd-b7ea-35531ef1adf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:32.771963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f2aab-fc85-41bd-b7ea-35531ef1adf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:32.773389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f2aab-fc85-41bd-b7ea-35531ef1adf4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-3586a273-cc08-4228-a158-461a19d5a554 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:32.775675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:32.776575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:32.777566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:32.777566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:32.777566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:32.777566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:32.777566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:32.777566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:32.777566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:32.777566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:32.777566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 18:04:32.777566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:32.784719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 431/856] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:04:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:32.786294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586a273-cc08-4228-a158-461a19d5a554 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:32.787324 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 426/857] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:32.793685 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f2aab-fc85-41bd-b7ea-35531ef1adf4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:32.814692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:32.899175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586a273-cc08-4228-a158-461a19d5a554 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:32.903752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f2aab-fc85-41bd-b7ea-35531ef1adf4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:32.941068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce04a40-33cb-48f5-ade3-234eced0377c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:32.941314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e5b7e-2a1f-4f00-86ef-dd538e2524f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:32.941767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04a40-33cb-48f5-ade3-234eced0377c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:32.941978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e5b7e-2a1f-4f00-86ef-dd538e2524f0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:32.942078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04a40-33cb-48f5-ade3-234eced0377c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:32.942264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e5b7e-2a1f-4f00-86ef-dd538e2524f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:32.942377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04a40-33cb-48f5-ade3-234eced0377c None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:32.942532 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e5b7e-2a1f-4f00-86ef-dd538e2524f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:32.943131 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce04a40-33cb-48f5-ade3-234eced0377c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:32.943357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e5b7e-2a1f-4f00-86ef-dd538e2524f0 None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=90dcc2061d6b4987a1722fab33875b0a) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:32.943452 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce04a40-33cb-48f5-ade3-234eced0377c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:32.943669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e5b7e-2a1f-4f00-86ef-dd538e2524f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:32.944934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce04a40-33cb-48f5-ade3-234eced0377c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:32.945421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e5b7e-2a1f-4f00-86ef-dd538e2524f0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:32.960313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 427/858] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:32 2022] GET /identity/v3/role_assignments?user.id=90dcc2061d6b4987a1722fab33875b0a&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:32.960313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3167e2e-d7ce-4826-921c-7089ed174b0c None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:32.960313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3167e2e-d7ce-4826-921c-7089ed174b0c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:32.960313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3167e2e-d7ce-4826-921c-7089ed174b0c None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:32.960313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 428/859] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:32.962273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95d354-7390-497c-a6d0-7d1ebfd1b175 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:32.962661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95d354-7390-497c-a6d0-7d1ebfd1b175 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:32.962937 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95d354-7390-497c-a6d0-7d1ebfd1b175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:32.969819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:32.978996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 432/860] 10.222.0.46 () {56 vars in 1135 bytes} [Thu May 26 18:04:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:32.982948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ef95d354-7390-497c-a6d0-7d1ebfd1b175 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:32.984148 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ef95d354-7390-497c-a6d0-7d1ebfd1b175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:32.986193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce04a40-33cb-48f5-ade3-234eced0377c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:33.086847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 429/861] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:33.111068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95d354-7390-497c-a6d0-7d1ebfd1b175 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:33.116138 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:33.132212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce04a40-33cb-48f5-ade3-234eced0377c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:33.172618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4afc96-ef0f-42f8-a48b-b5f118d1f937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:33.173309 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4afc96-ef0f-42f8-a48b-b5f118d1f937 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:33.173689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4afc96-ef0f-42f8-a48b-b5f118d1f937 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:33.174128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4afc96-ef0f-42f8-a48b-b5f118d1f937 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:33.240584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95d354-7390-497c-a6d0-7d1ebfd1b175 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:33.257210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4afc96-ef0f-42f8-a48b-b5f118d1f937 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:33.258895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4afc96-ef0f-42f8-a48b-b5f118d1f937 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:33.288400 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ef6fa-cfa5-42ef-8515-a5fde139604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:33.288883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ef6fa-cfa5-42ef-8515-a5fde139604a None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:33.289348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ef6fa-cfa5-42ef-8515-a5fde139604a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:33.289348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ef6fa-cfa5-42ef-8515-a5fde139604a None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:33.290132 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ef6fa-cfa5-42ef-8515-a5fde139604a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:33.291543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ef6fa-cfa5-42ef-8515-a5fde139604a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:33.294316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:33.312810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 430/862] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:04:33 2022] POST /identity/v3/services => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:04:33.370140 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:33.391773 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 433/863] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:32 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 18:04:33.397477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ef6fa-cfa5-42ef-8515-a5fde139604a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:33.551282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ef6fa-cfa5-42ef-8515-a5fde139604a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:33.589155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f844d-c6c5-497d-83e5-356885d11a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:33.590984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f844d-c6c5-497d-83e5-356885d11a89 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:33.591274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f844d-c6c5-497d-83e5-356885d11a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:33.591547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f844d-c6c5-497d-83e5-356885d11a89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:33.604408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937f844d-c6c5-497d-83e5-356885d11a89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:33.606510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937f844d-c6c5-497d-83e5-356885d11a89 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-937f844d-c6c5-497d-83e5-356885d11a89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:33.625687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:33.626670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:33.627580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:33.627580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:33.627580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:33.627580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:33.629936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 431/864] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:33.635986 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4afc96-ef0f-42f8-a48b-b5f118d1f937 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:33.720665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919b274-e9d3-48b3-ac14-0d5ae6fd0aec None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:33.721012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919b274-e9d3-48b3-ac14-0d5ae6fd0aec None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:33.721225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919b274-e9d3-48b3-ac14-0d5ae6fd0aec None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:33.722883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 432/865] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:33.728136 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d649b-037d-4f9b-93aa-f4154bb7c840 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:33.728417 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d649b-037d-4f9b-93aa-f4154bb7c840 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:33.728688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d649b-037d-4f9b-93aa-f4154bb7c840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:33.749791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4afc96-ef0f-42f8-a48b-b5f118d1f937 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:33.751737 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0e1d649b-037d-4f9b-93aa-f4154bb7c840 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:33.753939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0e1d649b-037d-4f9b-93aa-f4154bb7c840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:33.789842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e37e6b-c68b-4ad7-a6b1-3adb17edd319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:33.790543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e37e6b-c68b-4ad7-a6b1-3adb17edd319 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:33.790829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e37e6b-c68b-4ad7-a6b1-3adb17edd319 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:33.791108 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e37e6b-c68b-4ad7-a6b1-3adb17edd319 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:33.791832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e37e6b-c68b-4ad7-a6b1-3adb17edd319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:33.792141 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e37e6b-c68b-4ad7-a6b1-3adb17edd319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:33.794264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e37e6b-c68b-4ad7-a6b1-3adb17edd319 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:33.803018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:33.810679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 434/866] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:33.816337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e37e6b-c68b-4ad7-a6b1-3adb17edd319 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:33.859549 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 433/867] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:33.875077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:33.880965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d649b-037d-4f9b-93aa-f4154bb7c840 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:33.958620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e37e6b-c68b-4ad7-a6b1-3adb17edd319 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:34.001907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095c1c5-d767-44ba-8a20-c97fee72f738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:34.003322 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095c1c5-d767-44ba-8a20-c97fee72f738 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:34.003791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095c1c5-d767-44ba-8a20-c97fee72f738 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:34.004245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095c1c5-d767-44ba-8a20-c97fee72f738 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:34.012594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095c1c5-d767-44ba-8a20-c97fee72f738 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:34.013875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d649b-037d-4f9b-93aa-f4154bb7c840 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:34.014115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095c1c5-d767-44ba-8a20-c97fee72f738 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-7095c1c5-d767-44ba-8a20-c97fee72f738 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:34.034803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:34.036848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:34.038628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:34.038628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:34.038628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:34.038628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:34.038628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:34.038628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:34.038628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:34.038628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:34.041556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 435/868] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:34.043243 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095c1c5-d767-44ba-8a20-c97fee72f738 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:34.065184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b83164-4f59-43ba-bfb9-b232a0c611b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:34.066641 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b83164-4f59-43ba-bfb9-b232a0c611b1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:34.067221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b83164-4f59-43ba-bfb9-b232a0c611b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:34.067692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b83164-4f59-43ba-bfb9-b232a0c611b1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:34.068564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b83164-4f59-43ba-bfb9-b232a0c611b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:34.070353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b83164-4f59-43ba-bfb9-b232a0c611b1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-67b83164-4f59-43ba-bfb9-b232a0c611b1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:34.084666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:34.085682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 18:04:34.086533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:34.090040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 434/869] 10.222.0.46 () {56 vars in 1132 bytes} [Thu May 26 18:04:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:34.095806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b83164-4f59-43ba-bfb9-b232a0c611b1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:34.137761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:34.163049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:34.167074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095c1c5-d767-44ba-8a20-c97fee72f738 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:34.207721 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c64923-eb17-46b9-81b3-306bba1d7139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:34.208647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c64923-eb17-46b9-81b3-306bba1d7139 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:34.209079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c64923-eb17-46b9-81b3-306bba1d7139 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:34.209594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c64923-eb17-46b9-81b3-306bba1d7139 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:34.210719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c64923-eb17-46b9-81b3-306bba1d7139 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:34.211219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c64923-eb17-46b9-81b3-306bba1d7139 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:34.212945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c64923-eb17-46b9-81b3-306bba1d7139 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:34.217231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b83164-4f59-43ba-bfb9-b232a0c611b1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:34.231928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 436/870] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:34.233294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c64923-eb17-46b9-81b3-306bba1d7139 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:34.256086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4fbbd-d63a-4fa8-bcd2-a6f9af0e5a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:34.256758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4fbbd-d63a-4fa8-bcd2-a6f9af0e5a05 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:34.257020 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4fbbd-d63a-4fa8-bcd2-a6f9af0e5a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:34.257287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4fbbd-d63a-4fa8-bcd2-a6f9af0e5a05 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:34.258027 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4fbbd-d63a-4fa8-bcd2-a6f9af0e5a05 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:34.258327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4fbbd-d63a-4fa8-bcd2-a6f9af0e5a05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:34.259807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4fbbd-d63a-4fa8-bcd2-a6f9af0e5a05 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:34.270213 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 435/871] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 18:04:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:34.275660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4fbbd-d63a-4fa8-bcd2-a6f9af0e5a05 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:34.379361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c64923-eb17-46b9-81b3-306bba1d7139 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:34.400536 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:34.405877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4fbbd-d63a-4fa8-bcd2-a6f9af0e5a05 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:34.427844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b57ffa-e58d-455e-904e-9ab9a833983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:34.428765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b57ffa-e58d-455e-904e-9ab9a833983f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:34.429190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b57ffa-e58d-455e-904e-9ab9a833983f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:34.429680 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b57ffa-e58d-455e-904e-9ab9a833983f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:34.430815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b57ffa-e58d-455e-904e-9ab9a833983f None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=d656b42c2acb498caa92c77a22fd7efe) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:34.431315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b57ffa-e58d-455e-904e-9ab9a833983f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:34.433631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b57ffa-e58d-455e-904e-9ab9a833983f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:34.441273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 437/872] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:34 2022] GET /identity/v3/role_assignments?user.id=d656b42c2acb498caa92c77a22fd7efe&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=584beecb022b422f820e77f7623ada5e => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:34.456441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5912a-495b-409a-bd16-9026dbd06c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:34.457145 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5912a-495b-409a-bd16-9026dbd06c74 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:34.457427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5912a-495b-409a-bd16-9026dbd06c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:34.457699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5912a-495b-409a-bd16-9026dbd06c74 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:34.458756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5912a-495b-409a-bd16-9026dbd06c74 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:34.459064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5912a-495b-409a-bd16-9026dbd06c74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:34.460956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5912a-495b-409a-bd16-9026dbd06c74 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:34.469931 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 436/873] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 18:04:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:35.474171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f591199-84c7-470c-a70f-d1593c229f82 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:35.474738 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f591199-84c7-470c-a70f-d1593c229f82 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:35.474738 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f591199-84c7-470c-a70f-d1593c229f82 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:35.476409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 438/874] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:35.483374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a9413-c37f-4cb2-b24b-55594a6fcc21 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:35.483897 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a9413-c37f-4cb2-b24b-55594a6fcc21 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:35.483897 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a9413-c37f-4cb2-b24b-55594a6fcc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:35.503907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ec0a9413-c37f-4cb2-b24b-55594a6fcc21 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:35.505154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ec0a9413-c37f-4cb2-b24b-55594a6fcc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:35.613879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 437/875] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:35 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:35.638078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f591199-84c7-470c-a70f-d1593c229f82 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:35.668851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:35.705884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f5036-20a6-42b5-92a4-8bca9d6a9a5e None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:35.706223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f5036-20a6-42b5-92a4-8bca9d6a9a5e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:35.706456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f5036-20a6-42b5-92a4-8bca9d6a9a5e None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:35.708078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 438/876] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:35.713583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef965e-ae3d-44f9-a7f4-51d1a51ead65 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:35.714009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef965e-ae3d-44f9-a7f4-51d1a51ead65 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:35.714423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef965e-ae3d-44f9-a7f4-51d1a51ead65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:35.733113 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a2ef965e-ae3d-44f9-a7f4-51d1a51ead65 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:35.734604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a2ef965e-ae3d-44f9-a7f4-51d1a51ead65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:35.759328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f591199-84c7-470c-a70f-d1593c229f82 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:35.796290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8124215-3c9c-4ee0-9590-35fa238f8f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:35.797350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8124215-3c9c-4ee0-9590-35fa238f8f2d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:35.797803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8124215-3c9c-4ee0-9590-35fa238f8f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:35.798212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8124215-3c9c-4ee0-9590-35fa238f8f2d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:35.811386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 439/877] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:35 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:35.811895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8124215-3c9c-4ee0-9590-35fa238f8f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-e8124215-3c9c-4ee0-9590-35fa238f8f2d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:35.820347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:35.821271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:35.822152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:35.822152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:35.822152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:35.822152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:35.822152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:35.822152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:35.822152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:35.822152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:35.822152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:35.822152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:35.822152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 18:04:35.822152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:35.825556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 439/878] 10.222.0.46 () {56 vars in 1120 bytes} [Thu May 26 18:04:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:35.833775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef965e-ae3d-44f9-a7f4-51d1a51ead65 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:35.838054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8124215-3c9c-4ee0-9590-35fa238f8f2d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:35.858572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:35.935870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:35.953995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8124215-3c9c-4ee0-9590-35fa238f8f2d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:35.956734 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef965e-ae3d-44f9-a7f4-51d1a51ead65 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:35.993109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76267dbc-d19a-488c-bcff-53f224aee6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:35.993968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76267dbc-d19a-488c-bcff-53f224aee6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:35.994253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76267dbc-d19a-488c-bcff-53f224aee6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:35.994527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76267dbc-d19a-488c-bcff-53f224aee6e2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:35.995177 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76267dbc-d19a-488c-bcff-53f224aee6e2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:35.997134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76267dbc-d19a-488c-bcff-53f224aee6e2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:35.999340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f92293-7cc6-45c7-a551-994ffb27354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:36.000332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f92293-7cc6-45c7-a551-994ffb27354c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:36.000844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f92293-7cc6-45c7-a551-994ffb27354c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:36.001317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f92293-7cc6-45c7-a551-994ffb27354c None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:36.002295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f92293-7cc6-45c7-a551-994ffb27354c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:36.002817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f92293-7cc6-45c7-a551-994ffb27354c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:36.004497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f92293-7cc6-45c7-a551-994ffb27354c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-76267dbc-d19a-488c-bcff-53f224aee6e2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:36.009755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:36.010748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 18:04:36.012153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:36.016593 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 440/879] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:04:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:36.017559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76267dbc-d19a-488c-bcff-53f224aee6e2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:36.018061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 440/880] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 18:04:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:36.023810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f92293-7cc6-45c7-a551-994ffb27354c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:36.119484 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:36.134852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76267dbc-d19a-488c-bcff-53f224aee6e2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:36.143119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f92293-7cc6-45c7-a551-994ffb27354c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:36.172654 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75e06d-086f-462e-b738-8a8044dde86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:36.173572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75e06d-086f-462e-b738-8a8044dde86b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:36.174056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75e06d-086f-462e-b738-8a8044dde86b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:36.174496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75e06d-086f-462e-b738-8a8044dde86b None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:36.175361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75e06d-086f-462e-b738-8a8044dde86b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:36.175839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75e06d-086f-462e-b738-8a8044dde86b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:36.177394 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75e06d-086f-462e-b738-8a8044dde86b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:36.188085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 441/881] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:04:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:36.189274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75e06d-086f-462e-b738-8a8044dde86b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:36.190368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bb303-4c56-4097-bdae-b94c28b4d827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:36.191273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bb303-4c56-4097-bdae-b94c28b4d827 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:36.191653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bb303-4c56-4097-bdae-b94c28b4d827 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:36.192026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bb303-4c56-4097-bdae-b94c28b4d827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:36.206963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:36.209314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bb303-4c56-4097-bdae-b94c28b4d827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:36.212346 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bb303-4c56-4097-bdae-b94c28b4d827 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-297bb303-4c56-4097-bdae-b94c28b4d827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:36.236394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:36.237437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:36.238362 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:36.238362 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:36.238362 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:36.238362 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:36.238594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 441/882] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:36.243632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bb303-4c56-4097-bdae-b94c28b4d827 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:36.327163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75e06d-086f-462e-b738-8a8044dde86b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:36.355823 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bb303-4c56-4097-bdae-b94c28b4d827 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:36.367869 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7db86f-65a5-4bde-a9e2-d2185b2916da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:36.368909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db86f-65a5-4bde-a9e2-d2185b2916da None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:36.369399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db86f-65a5-4bde-a9e2-d2185b2916da None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:36.369923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db86f-65a5-4bde-a9e2-d2185b2916da None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:36.371504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7db86f-65a5-4bde-a9e2-d2185b2916da None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:36.372137 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7db86f-65a5-4bde-a9e2-d2185b2916da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:36.373776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7db86f-65a5-4bde-a9e2-d2185b2916da None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:36.384216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 442/883] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:04:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:36.385449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7db86f-65a5-4bde-a9e2-d2185b2916da None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:36.400278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb124a4-39a8-4b25-861b-0c46abb8d7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:36.401186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb124a4-39a8-4b25-861b-0c46abb8d7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:36.401626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb124a4-39a8-4b25-861b-0c46abb8d7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:36.402334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb124a4-39a8-4b25-861b-0c46abb8d7f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:36.403320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb124a4-39a8-4b25-861b-0c46abb8d7f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:36.403460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb124a4-39a8-4b25-861b-0c46abb8d7f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:36.404740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb124a4-39a8-4b25-861b-0c46abb8d7f6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:36.412515 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:36.424794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 442/884] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:36.426182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb124a4-39a8-4b25-861b-0c46abb8d7f6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:36.498382 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:36.522848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7db86f-65a5-4bde-a9e2-d2185b2916da None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:36.551452 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb124a4-39a8-4b25-861b-0c46abb8d7f6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:36.567517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e80df-bb45-458e-9845-e949915f9d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:36.568504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e80df-bb45-458e-9845-e949915f9d02 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:36.568793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e80df-bb45-458e-9845-e949915f9d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:36.569068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e80df-bb45-458e-9845-e949915f9d02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:36.569986 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e80df-bb45-458e-9845-e949915f9d02 None admin] RBAC: Adding query filter params (interface=public, service_id=1b894eaa405b4244bad707d92e9da12c, region_id=RegionOne) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:36.570303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e80df-bb45-458e-9845-e949915f9d02 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:36.572244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e80df-bb45-458e-9845-e949915f9d02 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:36.583564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 443/885] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 18:04:36 2022] GET /identity/v3/endpoints?service_id=1b894eaa405b4244bad707d92e9da12c&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:36.584954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e80df-bb45-458e-9845-e949915f9d02 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:36.601344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5905491-9efc-4f4c-b17b-5de955f3f465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:36.601922 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5905491-9efc-4f4c-b17b-5de955f3f465 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:36.602206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5905491-9efc-4f4c-b17b-5de955f3f465 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:36.602479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5905491-9efc-4f4c-b17b-5de955f3f465 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:36.611671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5905491-9efc-4f4c-b17b-5de955f3f465 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:36.613164 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5905491-9efc-4f4c-b17b-5de955f3f465 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e5905491-9efc-4f4c-b17b-5de955f3f465 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:36.630327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:36.631366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:36.632340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:36.632340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:36.632340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:36.632340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:36.632340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:36.632340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:36.632340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:36.632340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:36.632340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 18:04:36.632340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:36.636946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 443/886] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 18:04:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:36.638209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5905491-9efc-4f4c-b17b-5de955f3f465 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:36.702730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:36.713799 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e80df-bb45-458e-9845-e949915f9d02 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:36.747473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5905491-9efc-4f4c-b17b-5de955f3f465 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:36.754066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875966b-7c97-4d1c-a193-3482c5ec07f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:36.755128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875966b-7c97-4d1c-a193-3482c5ec07f1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:36.755620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875966b-7c97-4d1c-a193-3482c5ec07f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:36.756123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875966b-7c97-4d1c-a193-3482c5ec07f1 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:36.757037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875966b-7c97-4d1c-a193-3482c5ec07f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:36.758844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875966b-7c97-4d1c-a193-3482c5ec07f1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:36.770301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 444/887] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 18:04:36 2022] GET /identity/v3/services => generated 1011 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:36.791783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987e3c5-eeb8-4ed6-a127-99c50b4ca87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:36.793389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987e3c5-eeb8-4ed6-a127-99c50b4ca87c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:36.793575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044efdcf-9985-48a5-8b7f-7f502c8da12e None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:36.793675 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987e3c5-eeb8-4ed6-a127-99c50b4ca87c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:36.793911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044efdcf-9985-48a5-8b7f-7f502c8da12e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:36.794018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987e3c5-eeb8-4ed6-a127-99c50b4ca87c None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:36.794165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044efdcf-9985-48a5-8b7f-7f502c8da12e None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:36.794810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987e3c5-eeb8-4ed6-a127-99c50b4ca87c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:36.795126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987e3c5-eeb8-4ed6-a127-99c50b4ca87c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:36.796903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987e3c5-eeb8-4ed6-a127-99c50b4ca87c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:36.799840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 445/888] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:36.801326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964a02-e4c9-44a5-9d70-5d995cc79d53 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:36.801613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964a02-e4c9-44a5-9d70-5d995cc79d53 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:36.801936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964a02-e4c9-44a5-9d70-5d995cc79d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:36.815272 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:36.821402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-78964a02-e4c9-44a5-9d70-5d995cc79d53 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:36.822662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-78964a02-e4c9-44a5-9d70-5d995cc79d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:36.823235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 444/889] 10.222.0.46 () {56 vars in 1149 bytes} [Thu May 26 18:04:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:36.825081 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987e3c5-eeb8-4ed6-a127-99c50b4ca87c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:36.906302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 446/890] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:36 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:36.925409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78964a02-e4c9-44a5-9d70-5d995cc79d53 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:36.939570 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987e3c5-eeb8-4ed6-a127-99c50b4ca87c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:36.961582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:36.989589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3bc3b-73b4-42cb-b82e-afd128af6d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:36.990697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3bc3b-73b4-42cb-b82e-afd128af6d33 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:36.990996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3bc3b-73b4-42cb-b82e-afd128af6d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:36.991278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3bc3b-73b4-42cb-b82e-afd128af6d33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:37.073451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78964a02-e4c9-44a5-9d70-5d995cc79d53 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:37.082531 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3bc3b-73b4-42cb-b82e-afd128af6d33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:37.084114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3bc3b-73b4-42cb-b82e-afd128af6d33 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:37.112929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6f75d-cbf4-487a-b75f-244949f57326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:37.113915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6f75d-cbf4-487a-b75f-244949f57326 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:37.114199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6f75d-cbf4-487a-b75f-244949f57326 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:37.114474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6f75d-cbf4-487a-b75f-244949f57326 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:37.115138 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6f75d-cbf4-487a-b75f-244949f57326 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:37.116757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6f75d-cbf4-487a-b75f-244949f57326 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:37.119347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-18a6f75d-cbf4-487a-b75f-244949f57326 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:37.132694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:37.133803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:37.134645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 447/891] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 18:04:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:37.146786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6f75d-cbf4-487a-b75f-244949f57326 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:37.223262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 445/892] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:36 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 18:04:37.225438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a411fe-03e9-4ba4-bd7a-ce6532f88e57 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:37.225982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a411fe-03e9-4ba4-bd7a-ce6532f88e57 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:37.226531 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a411fe-03e9-4ba4-bd7a-ce6532f88e57 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:37.228053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 446/893] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:37.228959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a411fe-03e9-4ba4-bd7a-ce6532f88e57 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:37.249088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:37.283479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6f75d-cbf4-487a-b75f-244949f57326 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:37.324023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d59183-39e3-499a-9dd1-14374869f474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:37.324635 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d59183-39e3-499a-9dd1-14374869f474 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:37.325012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d59183-39e3-499a-9dd1-14374869f474 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:37.325202 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d59183-39e3-499a-9dd1-14374869f474 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:37.325917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d59183-39e3-499a-9dd1-14374869f474 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:37.326221 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d59183-39e3-499a-9dd1-14374869f474 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:37.327635 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d59183-39e3-499a-9dd1-14374869f474 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:37.333062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 448/894] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 18:04:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:37.335430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e821f8-ef97-4b82-8188-a00c9acf6662 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:37.336428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e821f8-ef97-4b82-8188-a00c9acf6662 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:37.337185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e821f8-ef97-4b82-8188-a00c9acf6662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:37.346449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a411fe-03e9-4ba4-bd7a-ce6532f88e57 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:37.355565 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-97e821f8-ef97-4b82-8188-a00c9acf6662 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:37.356907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-97e821f8-ef97-4b82-8188-a00c9acf6662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:37.384032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8236d8-733c-421d-8439-205ce1ebeed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:37.384817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8236d8-733c-421d-8439-205ce1ebeed3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:37.385088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8236d8-733c-421d-8439-205ce1ebeed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:37.385363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8236d8-733c-421d-8439-205ce1ebeed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:37.396335 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8236d8-733c-421d-8439-205ce1ebeed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:37.397793 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8236d8-733c-421d-8439-205ce1ebeed3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-bd8236d8-733c-421d-8439-205ce1ebeed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:37.416548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:37.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:37.419380 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:37.419380 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:37.419380 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:37.419380 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:37.421325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:37.423472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 447/895] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:37.424316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8236d8-733c-421d-8439-205ce1ebeed3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:37.434543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 449/896] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:37 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:37.435199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e821f8-ef97-4b82-8188-a00c9acf6662 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:37.505354 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:37.542564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8236d8-733c-421d-8439-205ce1ebeed3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:37.544859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e821f8-ef97-4b82-8188-a00c9acf6662 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:37.578795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4df55-be73-4852-a403-ccf7359bcf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:37.579446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4df55-be73-4852-a403-ccf7359bcf48 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:37.579689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4df55-be73-4852-a403-ccf7359bcf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:37.579928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4df55-be73-4852-a403-ccf7359bcf48 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:37.580627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4df55-be73-4852-a403-ccf7359bcf48 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:37.580826 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2452da7-3354-45ee-b7df-1466b10fd73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:37.580938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4df55-be73-4852-a403-ccf7359bcf48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:37.581482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2452da7-3354-45ee-b7df-1466b10fd73a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:37.581779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2452da7-3354-45ee-b7df-1466b10fd73a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:37.582048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2452da7-3354-45ee-b7df-1466b10fd73a None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:37.582318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4df55-be73-4852-a403-ccf7359bcf48 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:37.582757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2452da7-3354-45ee-b7df-1466b10fd73a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:37.583055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2452da7-3354-45ee-b7df-1466b10fd73a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:37.584428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2452da7-3354-45ee-b7df-1466b10fd73a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:37.587227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 448/897] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 18:04:37 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:37.595395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4df55-be73-4852-a403-ccf7359bcf48 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:37.603485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 450/898] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:37.604586 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2452da7-3354-45ee-b7df-1466b10fd73a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:37.676809 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:37.736290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4df55-be73-4852-a403-ccf7359bcf48 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:37.753816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2452da7-3354-45ee-b7df-1466b10fd73a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:37.776219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:37.784311 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2543528-fbe2-40ac-82d9-eab09c8cd0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:37.784994 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2543528-fbe2-40ac-82d9-eab09c8cd0e0 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:37.785280 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2543528-fbe2-40ac-82d9-eab09c8cd0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:37.785555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2543528-fbe2-40ac-82d9-eab09c8cd0e0 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:37.786863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2543528-fbe2-40ac-82d9-eab09c8cd0e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:37.788413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2543528-fbe2-40ac-82d9-eab09c8cd0e0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:37.803883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c8b5b-6db5-4e38-b5dc-3235aedb2644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:37.804725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c8b5b-6db5-4e38-b5dc-3235aedb2644 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:37.805003 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c8b5b-6db5-4e38-b5dc-3235aedb2644 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:37.805274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c8b5b-6db5-4e38-b5dc-3235aedb2644 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:37.807286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 449/899] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:04:37 2022] POST /identity/v3/services => generated 269 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:37.825866 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c8b5b-6db5-4e38-b5dc-3235aedb2644 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:37.827819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c8b5b-6db5-4e38-b5dc-3235aedb2644 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8b0c8b5b-6db5-4e38-b5dc-3235aedb2644 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:37.846004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:37.846971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:37.848022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:37.848022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:37.848022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:37.848022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:37.848022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:37.848022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:37.848022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:37.848022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:37.848022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 451/900] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:37.855482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2543528-fbe2-40ac-82d9-eab09c8cd0e0 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:37.994663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2543528-fbe2-40ac-82d9-eab09c8cd0e0 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:38.035476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606307-36c0-4ab6-8b5c-d23055b202d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:38.036452 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606307-36c0-4ab6-8b5c-d23055b202d9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.036787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606307-36c0-4ab6-8b5c-d23055b202d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.037073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606307-36c0-4ab6-8b5c-d23055b202d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.037957 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c606307-36c0-4ab6-8b5c-d23055b202d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:38.038271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c606307-36c0-4ab6-8b5c-d23055b202d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:38.039965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c606307-36c0-4ab6-8b5c-d23055b202d9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:38.057423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 450/901] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:38.063022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c8b5b-6db5-4e38-b5dc-3235aedb2644 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:38.187756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c8b5b-6db5-4e38-b5dc-3235aedb2644 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:38.202377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:38.226802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03531777-fea6-4b51-a55f-e90c35854025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:38.227997 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03531777-fea6-4b51-a55f-e90c35854025 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.228495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03531777-fea6-4b51-a55f-e90c35854025 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.228983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03531777-fea6-4b51-a55f-e90c35854025 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.231208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03531777-fea6-4b51-a55f-e90c35854025 None admin] RBAC: Adding query filter params (role.id=e9715685617b47c7809732b0a8c61c35, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=90dcc2061d6b4987a1722fab33875b0a) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:38.231766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03531777-fea6-4b51-a55f-e90c35854025 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:38.234445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03531777-fea6-4b51-a55f-e90c35854025 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:38.245507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 452/902] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:38 2022] GET /identity/v3/role_assignments?user.id=90dcc2061d6b4987a1722fab33875b0a&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=e9715685617b47c7809732b0a8c61c35 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:39.255636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00996126-6853-4cdd-a536-fa0ba870139d None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.256162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00996126-6853-4cdd-a536-fa0ba870139d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.256505 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00996126-6853-4cdd-a536-fa0ba870139d None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.259234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 451/903] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:39.268253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc216f-428f-4d0d-9d2d-97c71096b9fd None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.268731 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc216f-428f-4d0d-9d2d-97c71096b9fd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.268807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc216f-428f-4d0d-9d2d-97c71096b9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.289764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e5cc216f-428f-4d0d-9d2d-97c71096b9fd None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:39.290970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e5cc216f-428f-4d0d-9d2d-97c71096b9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:39.366615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 453/904] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:39.391340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00996126-6853-4cdd-a536-fa0ba870139d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.451201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:39.508768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00996126-6853-4cdd-a536-fa0ba870139d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.547620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98f3de-8122-4c9a-85e5-54a25f900319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.548437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98f3de-8122-4c9a-85e5-54a25f900319 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.548721 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98f3de-8122-4c9a-85e5-54a25f900319 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.548998 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98f3de-8122-4c9a-85e5-54a25f900319 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.549654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98f3de-8122-4c9a-85e5-54a25f900319 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.551074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98f3de-8122-4c9a-85e5-54a25f900319 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-9d98f3de-8122-4c9a-85e5-54a25f900319 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:39.564363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:39.565423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:39.566296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 452/905] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:39.571798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc216f-428f-4d0d-9d2d-97c71096b9fd None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.686004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc216f-428f-4d0d-9d2d-97c71096b9fd None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.694304 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:39.731304 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9202e9a-45d1-4281-bb8b-7bfec23de14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.732242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202e9a-45d1-4281-bb8b-7bfec23de14e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.732521 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202e9a-45d1-4281-bb8b-7bfec23de14e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.732795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202e9a-45d1-4281-bb8b-7bfec23de14e None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.733506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9202e9a-45d1-4281-bb8b-7bfec23de14e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:39.733902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9202e9a-45d1-4281-bb8b-7bfec23de14e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.735298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9202e9a-45d1-4281-bb8b-7bfec23de14e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.743691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 454/906] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:39.750431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98f3de-8122-4c9a-85e5-54a25f900319 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.876466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98f3de-8122-4c9a-85e5-54a25f900319 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.891302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:39.922444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4d011-1532-48ba-b10b-6420042d9a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.923117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4d011-1532-48ba-b10b-6420042d9a8f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.923397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4d011-1532-48ba-b10b-6420042d9a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.923672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4d011-1532-48ba-b10b-6420042d9a8f None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.924377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4d011-1532-48ba-b10b-6420042d9a8f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:39.924682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4d011-1532-48ba-b10b-6420042d9a8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.926095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4d011-1532-48ba-b10b-6420042d9a8f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.933571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 453/907] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:39.941095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9202e9a-45d1-4281-bb8b-7bfec23de14e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.007673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b50a1-12df-4901-ab6f-bcbb10447857 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.008003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b50a1-12df-4901-ab6f-bcbb10447857 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.008214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b50a1-12df-4901-ab6f-bcbb10447857 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.009976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 454/908] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:40.017175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407b196-e252-4bcc-be53-6c85a45b6854 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.017831 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407b196-e252-4bcc-be53-6c85a45b6854 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.018372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407b196-e252-4bcc-be53-6c85a45b6854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.041303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e407b196-e252-4bcc-be53-6c85a45b6854 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:40.042536 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e407b196-e252-4bcc-be53-6c85a45b6854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:40.058671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9202e9a-45d1-4281-bb8b-7bfec23de14e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.101277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172db17d-4256-4cd1-8f03-8fc53aadc3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.102090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172db17d-4256-4cd1-8f03-8fc53aadc3d6 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.102425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172db17d-4256-4cd1-8f03-8fc53aadc3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.102702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172db17d-4256-4cd1-8f03-8fc53aadc3d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.103460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172db17d-4256-4cd1-8f03-8fc53aadc3d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.105046 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172db17d-4256-4cd1-8f03-8fc53aadc3d6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.117225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 455/909] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:40.133435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 455/910] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:40.141481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407b196-e252-4bcc-be53-6c85a45b6854 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.268591 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407b196-e252-4bcc-be53-6c85a45b6854 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.278415 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:40.316908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9ebaa-440c-4494-a5c7-4e47a6673636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.317649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9ebaa-440c-4494-a5c7-4e47a6673636 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.317966 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9ebaa-440c-4494-a5c7-4e47a6673636 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.318240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9ebaa-440c-4494-a5c7-4e47a6673636 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.318973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9ebaa-440c-4494-a5c7-4e47a6673636 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.320479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9ebaa-440c-4494-a5c7-4e47a6673636 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.339419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 456/911] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:04:40 2022] POST /identity/v3/services => generated 241 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:40.423830 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6f956-167f-4471-a614-faddf313ae88 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.424377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6f956-167f-4471-a614-faddf313ae88 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.424377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6f956-167f-4471-a614-faddf313ae88 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.426293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 456/912] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:40.432755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d422b-d685-42ab-b0b9-5181262bfb9f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.433650 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d422b-d685-42ab-b0b9-5181262bfb9f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.434229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d422b-d685-42ab-b0b9-5181262bfb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.453302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-919d422b-d685-42ab-b0b9-5181262bfb9f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:40.454980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-919d422b-d685-42ab-b0b9-5181262bfb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:40.561754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 457/913] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:40.581372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6f956-167f-4471-a614-faddf313ae88 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.630100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:40.639604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62ffed-ffd6-496c-bd3f-932b5dff18f8 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.639884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62ffed-ffd6-496c-bd3f-932b5dff18f8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.640324 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62ffed-ffd6-496c-bd3f-932b5dff18f8 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.642245 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 458/914] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:40.653756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d2fdf-c54b-458e-98ae-5ce7ef0a3321 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.653756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d2fdf-c54b-458e-98ae-5ce7ef0a3321 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.653756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d2fdf-c54b-458e-98ae-5ce7ef0a3321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.686867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-882d2fdf-c54b-458e-98ae-5ce7ef0a3321 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:40.688423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-882d2fdf-c54b-458e-98ae-5ce7ef0a3321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:40.738652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6f956-167f-4471-a614-faddf313ae88 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.822809 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1822c39-99bd-4d6f-9488-ee6e7e23f058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.825552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1822c39-99bd-4d6f-9488-ee6e7e23f058 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.825552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1822c39-99bd-4d6f-9488-ee6e7e23f058 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.825552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1822c39-99bd-4d6f-9488-ee6e7e23f058 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.826169 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1822c39-99bd-4d6f-9488-ee6e7e23f058 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.827975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1822c39-99bd-4d6f-9488-ee6e7e23f058 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.842164 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 459/915] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-d1822c39-99bd-4d6f-9488-ee6e7e23f058 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:40.845368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:40.846320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 18:04:40.847142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:40.849041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 457/916] 10.222.0.46 () {56 vars in 1132 bytes} [Thu May 26 18:04:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:40.857940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d2fdf-c54b-458e-98ae-5ce7ef0a3321 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.883736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1822c39-99bd-4d6f-9488-ee6e7e23f058 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.895228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:41.018368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:41.034525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1822c39-99bd-4d6f-9488-ee6e7e23f058 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.080103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d2fdf-c54b-458e-98ae-5ce7ef0a3321 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.126569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbde34-2f3d-4775-81a9-19d951054741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.127388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbde34-2f3d-4775-81a9-19d951054741 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.127734 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbde34-2f3d-4775-81a9-19d951054741 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.128054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbde34-2f3d-4775-81a9-19d951054741 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.147833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbde34-2f3d-4775-81a9-19d951054741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:41.149509 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24029b35-cb43-4da2-a229-511a5c03f174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.151296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24029b35-cb43-4da2-a229-511a5c03f174 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.152633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24029b35-cb43-4da2-a229-511a5c03f174 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.154008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24029b35-cb43-4da2-a229-511a5c03f174 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.156730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24029b35-cb43-4da2-a229-511a5c03f174 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:41.157267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24029b35-cb43-4da2-a229-511a5c03f174 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.160684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24029b35-cb43-4da2-a229-511a5c03f174 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a2cbde34-2f3d-4775-81a9-19d951054741 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:41.165420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:41.166369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:41.167280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:41.167280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:41.167280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:41.167280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:41.167280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:41.167280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:41.167280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:41.167280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:41.167280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:41.167280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:41.167280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 18:04:41.167280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:41.168412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 460/917] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 18:04:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:41.170390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 458/918] 10.222.0.46 () {56 vars in 1120 bytes} [Thu May 26 18:04:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:41.176619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24029b35-cb43-4da2-a229-511a5c03f174 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.176900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbde34-2f3d-4775-81a9-19d951054741 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.300043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24029b35-cb43-4da2-a229-511a5c03f174 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.314196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:41.342369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b80be-714f-4108-88e6-befb531e2eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.343535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b80be-714f-4108-88e6-befb531e2eea None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.344035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b80be-714f-4108-88e6-befb531e2eea None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.344506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b80be-714f-4108-88e6-befb531e2eea None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.345559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b80be-714f-4108-88e6-befb531e2eea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:41.346113 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b80be-714f-4108-88e6-befb531e2eea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.347913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b80be-714f-4108-88e6-befb531e2eea None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.351297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbde34-2f3d-4775-81a9-19d951054741 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.363786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 461/919] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 18:04:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:41.369262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b80be-714f-4108-88e6-befb531e2eea None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.395255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfea55-8dfd-4e76-93bd-ab3c4839b8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.396219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfea55-8dfd-4e76-93bd-ab3c4839b8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.396641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfea55-8dfd-4e76-93bd-ab3c4839b8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.397052 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfea55-8dfd-4e76-93bd-ab3c4839b8a4 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.397937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfea55-8dfd-4e76-93bd-ab3c4839b8a4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:41.398381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfea55-8dfd-4e76-93bd-ab3c4839b8a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.399935 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfea55-8dfd-4e76-93bd-ab3c4839b8a4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.415559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 459/920] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 18:04:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:41.417087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfea55-8dfd-4e76-93bd-ab3c4839b8a4 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.440984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:41.515375 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b80be-714f-4108-88e6-befb531e2eea None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.539276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfea55-8dfd-4e76-93bd-ab3c4839b8a4 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.564944 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce5ab-c1a1-42c3-8b19-a46d6b6d733c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.566139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce5ab-c1a1-42c3-8b19-a46d6b6d733c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.566621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce5ab-c1a1-42c3-8b19-a46d6b6d733c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.567087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce5ab-c1a1-42c3-8b19-a46d6b6d733c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.577893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d01379-27c9-4a95-8310-d5afb0b95e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.578798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d01379-27c9-4a95-8310-d5afb0b95e07 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.579251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d01379-27c9-4a95-8310-d5afb0b95e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.579662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d01379-27c9-4a95-8310-d5afb0b95e07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.580593 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d01379-27c9-4a95-8310-d5afb0b95e07 None admin] RBAC: Adding query filter params (interface=public, service_id=3832bc45e7124fb68e4a33c73adb0901, region_id=RegionOne) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:41.581123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d01379-27c9-4a95-8310-d5afb0b95e07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.582081 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ce5ab-c1a1-42c3-8b19-a46d6b6d733c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.582871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d01379-27c9-4a95-8310-d5afb0b95e07 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.584737 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ce5ab-c1a1-42c3-8b19-a46d6b6d733c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.590260 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:41.594348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 460/921] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 18:04:41 2022] GET /identity/v3/endpoints?service_id=3832bc45e7124fb68e4a33c73adb0901&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:41.596116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d01379-27c9-4a95-8310-d5afb0b95e07 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-1e5ce5ab-c1a1-42c3-8b19-a46d6b6d733c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:41.610978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:41.611910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:41.612894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:41.612894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:41.612894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:41.612894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:41.620514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 462/922] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:41.622250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce5ab-c1a1-42c3-8b19-a46d6b6d733c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.697615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:41.730057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d01379-27c9-4a95-8310-d5afb0b95e07 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.738976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce5ab-c1a1-42c3-8b19-a46d6b6d733c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.771521 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e074dd-4f76-40ed-9ab0-84468a54dd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.772144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e074dd-4f76-40ed-9ab0-84468a54dd61 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.772425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e074dd-4f76-40ed-9ab0-84468a54dd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.772699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e074dd-4f76-40ed-9ab0-84468a54dd61 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.773438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e074dd-4f76-40ed-9ab0-84468a54dd61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.774916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e074dd-4f76-40ed-9ab0-84468a54dd61 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.782625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 461/923] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 18:04:41 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:04:41.784407 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba8db5-74d8-47f2-8828-a8a5a6189016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.785303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba8db5-74d8-47f2-8828-a8a5a6189016 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.785859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba8db5-74d8-47f2-8828-a8a5a6189016 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.786601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba8db5-74d8-47f2-8828-a8a5a6189016 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.787552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba8db5-74d8-47f2-8828-a8a5a6189016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:41.788064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba8db5-74d8-47f2-8828-a8a5a6189016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.789707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba8db5-74d8-47f2-8828-a8a5a6189016 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.805484 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 463/924] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:41.812579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e074dd-4f76-40ed-9ab0-84468a54dd61 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.941328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e074dd-4f76-40ed-9ab0-84468a54dd61 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.982042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c4ce3-2564-4401-a472-6aa8db8252ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.982867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c4ce3-2564-4401-a472-6aa8db8252ff None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.983146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c4ce3-2564-4401-a472-6aa8db8252ff None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.983416 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c4ce3-2564-4401-a472-6aa8db8252ff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.994205 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c4ce3-2564-4401-a472-6aa8db8252ff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.995793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c4ce3-2564-4401-a472-6aa8db8252ff None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-651c4ce3-2564-4401-a472-6aa8db8252ff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:42.012472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:42.013638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:42.014602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:42.014602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:42.014602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:42.014602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:42.014602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:42.014602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:42.014602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:42.014602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:42.014602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 18:04:42.014602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:42.014602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 462/925] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 18:04:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:42.021667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba8db5-74d8-47f2-8828-a8a5a6189016 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:42.087818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:42.137564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba8db5-74d8-47f2-8828-a8a5a6189016 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:42.175018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f6ab5-2042-4371-b43b-3fc1d1c54a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:42.175699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f6ab5-2042-4371-b43b-3fc1d1c54a94 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.176002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f6ab5-2042-4371-b43b-3fc1d1c54a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.176255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f6ab5-2042-4371-b43b-3fc1d1c54a94 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.177087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f6ab5-2042-4371-b43b-3fc1d1c54a94 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:42.177393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f6ab5-2042-4371-b43b-3fc1d1c54a94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:42.178856 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f6ab5-2042-4371-b43b-3fc1d1c54a94 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.198440 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 464/926] 10.222.0.46 () {56 vars in 1149 bytes} [Thu May 26 18:04:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:42.203884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c4ce3-2564-4401-a472-6aa8db8252ff None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:42.207072 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:42.318463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c4ce3-2564-4401-a472-6aa8db8252ff None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:42.355849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb6915-fc06-440b-b9b5-39ee154dff94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:42.357033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb6915-fc06-440b-b9b5-39ee154dff94 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.357454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb6915-fc06-440b-b9b5-39ee154dff94 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.357895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb6915-fc06-440b-b9b5-39ee154dff94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.369585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb6915-fc06-440b-b9b5-39ee154dff94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:42.371572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb6915-fc06-440b-b9b5-39ee154dff94 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-66eb6915-fc06-440b-b9b5-39ee154dff94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:42.391656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:42.392795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:42.393942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:42.393942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:42.393942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:42.393942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:42.396035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 463/927] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:42.404015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f6ab5-2042-4371-b43b-3fc1d1c54a94 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:42.519921 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f6ab5-2042-4371-b43b-3fc1d1c54a94 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:42.543110 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:42.565251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf278f-5bfa-4da2-827e-a678922db861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:42.565955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf278f-5bfa-4da2-827e-a678922db861 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.566239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf278f-5bfa-4da2-827e-a678922db861 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.566512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf278f-5bfa-4da2-827e-a678922db861 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.567251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf278f-5bfa-4da2-827e-a678922db861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:42.567557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf278f-5bfa-4da2-827e-a678922db861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:42.568930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf278f-5bfa-4da2-827e-a678922db861 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.582451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 465/928] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:42.588054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb6915-fc06-440b-b9b5-39ee154dff94 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:42.661399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:42.715657 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb6915-fc06-440b-b9b5-39ee154dff94 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:42.757921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f5298-69c6-4bc5-ac40-3a1dafa3a307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:42.758763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f5298-69c6-4bc5-ac40-3a1dafa3a307 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.759040 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f5298-69c6-4bc5-ac40-3a1dafa3a307 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.759351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f5298-69c6-4bc5-ac40-3a1dafa3a307 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.767943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f5298-69c6-4bc5-ac40-3a1dafa3a307 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:42.769431 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f5298-69c6-4bc5-ac40-3a1dafa3a307 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-7f0f5298-69c6-4bc5-ac40-3a1dafa3a307 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:42.785023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:42.786306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:42.787285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:42.787285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:42.787285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:42.787285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:42.787285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:42.787285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:42.787285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:42.787285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:42.787285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 464/929] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:42.792919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf278f-5bfa-4da2-827e-a678922db861 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:42.832814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52cceb-286f-4e9a-96cf-44c194f5f146 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.833168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52cceb-286f-4e9a-96cf-44c194f5f146 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.833381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52cceb-286f-4e9a-96cf-44c194f5f146 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.835012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 465/930] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:42.840662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4407593-1ff2-43a6-9374-a7f48810deb8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.840943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4407593-1ff2-43a6-9374-a7f48810deb8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.841217 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4407593-1ff2-43a6-9374-a7f48810deb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.861778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d4407593-1ff2-43a6-9374-a7f48810deb8 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:42.863263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d4407593-1ff2-43a6-9374-a7f48810deb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:42.908787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf278f-5bfa-4da2-827e-a678922db861 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:42.952203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312607c-ef8c-4ac4-a5cf-b1a6464ef976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:42.953372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312607c-ef8c-4ac4-a5cf-b1a6464ef976 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.953983 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312607c-ef8c-4ac4-a5cf-b1a6464ef976 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.954434 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312607c-ef8c-4ac4-a5cf-b1a6464ef976 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.955927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312607c-ef8c-4ac4-a5cf-b1a6464ef976 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:42.956406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312607c-ef8c-4ac4-a5cf-b1a6464ef976 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:42.958117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312607c-ef8c-4ac4-a5cf-b1a6464ef976 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.959302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 466/931] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:42.960641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c5dc5-2dcb-471e-880c-bab22aca252e None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.965779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c5dc5-2dcb-471e-880c-bab22aca252e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.965779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c5dc5-2dcb-471e-880c-bab22aca252e None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.965779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 467/932] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:42.971999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a695ef-65ec-4875-a7bd-8ee78bad9103 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.972464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a695ef-65ec-4875-a7bd-8ee78bad9103 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.972889 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a695ef-65ec-4875-a7bd-8ee78bad9103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.980773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 466/933] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:42.986656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312607c-ef8c-4ac4-a5cf-b1a6464ef976 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:42.998633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c0a695ef-65ec-4875-a7bd-8ee78bad9103 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:42.999819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c0a695ef-65ec-4875-a7bd-8ee78bad9103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:43.029744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:43.042145 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:43.072229 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 468/934] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:43.073942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a695ef-65ec-4875-a7bd-8ee78bad9103 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:43.120713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312607c-ef8c-4ac4-a5cf-b1a6464ef976 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:43.160753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42989a0-b320-4a8e-842d-e9f83b5027c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:43.161941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42989a0-b320-4a8e-842d-e9f83b5027c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:43.162224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42989a0-b320-4a8e-842d-e9f83b5027c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:43.162498 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42989a0-b320-4a8e-842d-e9f83b5027c7 None admin] PATH_INFO: `/v3/projects/4be4045293754773bb52bbb82546a1db/users/90dcc2061d6b4987a1722fab33875b0a/roles/e9715685617b47c7809732b0a8c61c35` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:43.165305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42989a0-b320-4a8e-842d-e9f83b5027c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4be4045293754773bb52bbb82546a1db, user_id=90dcc2061d6b4987a1722fab33875b0a, role_id=e9715685617b47c7809732b0a8c61c35)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:43.167743 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42989a0-b320-4a8e-842d-e9f83b5027c7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:43.185534 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 467/935] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:04:42 2022] PUT /identity/v3/projects/4be4045293754773bb52bbb82546a1db/users/90dcc2061d6b4987a1722fab33875b0a/roles/e9715685617b47c7809732b0a8c61c35 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:04:43.193189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a695ef-65ec-4875-a7bd-8ee78bad9103 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:43.198851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42989a0-b320-4a8e-842d-e9f83b5027c7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:43.241354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfef2e-39eb-4ccb-a137-1512c346f5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:43.242425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfef2e-39eb-4ccb-a137-1512c346f5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:43.243280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfef2e-39eb-4ccb-a137-1512c346f5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:43.243280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfef2e-39eb-4ccb-a137-1512c346f5f1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:43.243941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfef2e-39eb-4ccb-a137-1512c346f5f1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:43.246127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfef2e-39eb-4ccb-a137-1512c346f5f1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-90dfef2e-39eb-4ccb-a137-1512c346f5f1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:43.262606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:43.264271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:43.265640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 469/936] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 18:04:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:43.273831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfef2e-39eb-4ccb-a137-1512c346f5f1 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:43.360836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:43.375343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:43.385531 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42989a0-b320-4a8e-842d-e9f83b5027c7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:43.441839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aab55-ae38-4464-b6d9-49e9d6bb1f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:43.442622 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aab55-ae38-4464-b6d9-49e9d6bb1f34 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:43.442797 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aab55-ae38-4464-b6d9-49e9d6bb1f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:43.443044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aab55-ae38-4464-b6d9-49e9d6bb1f34 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:43.444207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36aab55-ae38-4464-b6d9-49e9d6bb1f34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:43.446827 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36aab55-ae38-4464-b6d9-49e9d6bb1f34 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:43.449340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: WARNING keystone.server.flask.common [None req-b36aab55-ae38-4464-b6d9-49e9d6bb1f34 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 18:04:43.454251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfef2e-39eb-4ccb-a137-1512c346f5f1 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:43.471493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.sql.core [None req-b36aab55-ae38-4464-b6d9-49e9d6bb1f34 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 18:04:43.471493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 18:04:43.471493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [parameters: {'id': 'e4cac6d1b8584b9087fa128723e0984a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 18:04:43.471493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80324) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b36aab55-ae38-4464-b6d9-49e9d6bb1f34 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e4cac6d1b8584b9087fa128723e0984a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e4cac6d1b8584b9087fa128723e0984a. May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application result = self._query(query) May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application conn.query(q) May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 18:04:43.483039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application result.read() May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 26 18:04:43.484093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return project_ref.to_dict() May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application next(self.gen) May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application yield resource May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application next(self.gen) May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self.session.rollback() May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self.force_reraise() May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise self.value May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application session.commit() May 26 18:04:43.485238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._prepare_impl() May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self.session.flush() May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._flush(objects) May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application compat.raise_( May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application flush_context.execute() May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 18:04:43.486340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rec.execute(self) May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 18:04:43.487426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application result = self._query(query) May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application conn.query(q) May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application result.read() May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 18:04:43.488395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [parameters: {'id': 'e4cac6d1b8584b9087fa128723e0984a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:43.489461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:43.491366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:43.494162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:43.494162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 26 18:04:43.494162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 26 18:04:43.494162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:43.494162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:43.494162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 26 18:04:43.494162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.Conflict( May 26 18:04:43.494162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e4cac6d1b8584b9087fa128723e0984a. May 26 18:04:43.494162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:43.494162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 468/937] 10.222.0.46 () {60 vars in 1140 bytes} [Thu May 26 18:04:43 2022] POST /identity/v3/projects => generated 262 bytes in 295 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:43.499379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aab55-ae38-4464-b6d9-49e9d6bb1f34 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:43.509415 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f783b9-61fb-417b-8d9c-36a4f97c889b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:43.510540 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f783b9-61fb-417b-8d9c-36a4f97c889b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:43.511591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f783b9-61fb-417b-8d9c-36a4f97c889b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:43.512074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f783b9-61fb-417b-8d9c-36a4f97c889b None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:43.512990 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f783b9-61fb-417b-8d9c-36a4f97c889b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:43.513517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f783b9-61fb-417b-8d9c-36a4f97c889b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:43.515128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f783b9-61fb-417b-8d9c-36a4f97c889b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:43.522131 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 470/938] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 18:04:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:43.530853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f783b9-61fb-417b-8d9c-36a4f97c889b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:43.646604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aab55-ae38-4464-b6d9-49e9d6bb1f34 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:43.681414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f783b9-61fb-417b-8d9c-36a4f97c889b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:43.685710 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:43.698616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da9b51-e50f-4fbc-8104-3db3819348ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:43.699935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da9b51-e50f-4fbc-8104-3db3819348ee None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:43.700343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da9b51-e50f-4fbc-8104-3db3819348ee None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:43.701410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da9b51-e50f-4fbc-8104-3db3819348ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:43.709381 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da9b51-e50f-4fbc-8104-3db3819348ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:43.711415 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da9b51-e50f-4fbc-8104-3db3819348ee None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-c6da9b51-e50f-4fbc-8104-3db3819348ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:43.727918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:43.729026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:43.730442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:43.730442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:43.730442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:43.730442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:43.730442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:43.730442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:43.730442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:43.730442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:43.732121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 469/939] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:43.733601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6cb18-b08b-48af-a4ba-d69bffb0bd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:43.734318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6cb18-b08b-48af-a4ba-d69bffb0bd4b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:43.734597 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6cb18-b08b-48af-a4ba-d69bffb0bd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:43.734867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6cb18-b08b-48af-a4ba-d69bffb0bd4b None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:43.735606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6cb18-b08b-48af-a4ba-d69bffb0bd4b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:43.735911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6cb18-b08b-48af-a4ba-d69bffb0bd4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:43.737536 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da9b51-e50f-4fbc-8104-3db3819348ee None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:43.738539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6cb18-b08b-48af-a4ba-d69bffb0bd4b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:43.748870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 471/940] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 18:04:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:43.753712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:43.754638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6cb18-b08b-48af-a4ba-d69bffb0bd4b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:43.861030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da9b51-e50f-4fbc-8104-3db3819348ee None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:43.887530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6cb18-b08b-48af-a4ba-d69bffb0bd4b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:43.898792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8a143-c54e-45e7-bc01-7c0e2be91d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:43.899761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8a143-c54e-45e7-bc01-7c0e2be91d20 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:43.900248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8a143-c54e-45e7-bc01-7c0e2be91d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:43.900896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8a143-c54e-45e7-bc01-7c0e2be91d20 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:43.902396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8a143-c54e-45e7-bc01-7c0e2be91d20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:43.902907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8a143-c54e-45e7-bc01-7c0e2be91d20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:43.904752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8a143-c54e-45e7-bc01-7c0e2be91d20 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:43.933756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a03dd-10cd-4bad-ab41-d3d71daf6f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:43.933756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a03dd-10cd-4bad-ab41-d3d71daf6f14 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:43.933756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a03dd-10cd-4bad-ab41-d3d71daf6f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:43.933756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a03dd-10cd-4bad-ab41-d3d71daf6f14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:43.934199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a03dd-10cd-4bad-ab41-d3d71daf6f14 None admin] RBAC: Adding query filter params (interface=public, service_id=90054f6df98f4562adf8441146fd1248, region_id=RegionOne) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:43.934199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a03dd-10cd-4bad-ab41-d3d71daf6f14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:43.934199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a03dd-10cd-4bad-ab41-d3d71daf6f14 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:43.934984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 470/941] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:43.939079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 472/942] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 18:04:43 2022] GET /identity/v3/endpoints?service_id=90054f6df98f4562adf8441146fd1248&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:43.944895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8a143-c54e-45e7-bc01-7c0e2be91d20 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:44.055012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:44.069259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8a143-c54e-45e7-bc01-7c0e2be91d20 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:44.114560 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7ae16-be59-443e-8e37-117479170e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:44.115519 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7ae16-be59-443e-8e37-117479170e10 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:44.115950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7ae16-be59-443e-8e37-117479170e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:44.116397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7ae16-be59-443e-8e37-117479170e10 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:44.117856 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7ae16-be59-443e-8e37-117479170e10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:44.119427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7ae16-be59-443e-8e37-117479170e10 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:44.127589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 471/943] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 18:04:43 2022] GET /identity/v3/services => generated 1497 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:04:44.233825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2992d-e555-49b0-8290-903ece0f40b9 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:44.234570 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2992d-e555-49b0-8290-903ece0f40b9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:44.235065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2992d-e555-49b0-8290-903ece0f40b9 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:44.237127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 473/944] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:44.242887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8912e63-163b-4a69-84a7-a0afd5ae98a4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:44.243174 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8912e63-163b-4a69-84a7-a0afd5ae98a4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:44.243457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8912e63-163b-4a69-84a7-a0afd5ae98a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:44.266460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d8912e63-163b-4a69-84a7-a0afd5ae98a4 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:44.268011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d8912e63-163b-4a69-84a7-a0afd5ae98a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:44.347127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 472/945] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:44.381425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2992d-e555-49b0-8290-903ece0f40b9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:44.418248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:44.500352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2992d-e555-49b0-8290-903ece0f40b9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:44.538958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647ab9a-bf43-44f4-84a1-da3f776230b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:44.539773 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647ab9a-bf43-44f4-84a1-da3f776230b1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:44.540053 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647ab9a-bf43-44f4-84a1-da3f776230b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:44.540322 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647ab9a-bf43-44f4-84a1-da3f776230b1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:44.540971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647ab9a-bf43-44f4-84a1-da3f776230b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:44.542464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647ab9a-bf43-44f4-84a1-da3f776230b1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-1647ab9a-bf43-44f4-84a1-da3f776230b1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:44.555591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:44.556596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 18:04:44.557449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:44.563088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 474/946] 10.222.0.46 () {56 vars in 1132 bytes} [Thu May 26 18:04:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:44.564126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8912e63-163b-4a69-84a7-a0afd5ae98a4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:44.631745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:44.675649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8912e63-163b-4a69-84a7-a0afd5ae98a4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:44.715734 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4148793-02be-453d-b1f7-68aaad1a8a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:44.716776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4148793-02be-453d-b1f7-68aaad1a8a32 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:44.717274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4148793-02be-453d-b1f7-68aaad1a8a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:44.717768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4148793-02be-453d-b1f7-68aaad1a8a32 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:44.718687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4148793-02be-453d-b1f7-68aaad1a8a32 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:44.719184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4148793-02be-453d-b1f7-68aaad1a8a32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:44.720738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4148793-02be-453d-b1f7-68aaad1a8a32 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:44.726402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 473/947] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 18:04:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:44.733765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647ab9a-bf43-44f4-84a1-da3f776230b1 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:44.833993 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:44.841067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647ab9a-bf43-44f4-84a1-da3f776230b1 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:44.878461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdb823-d5b1-48b7-adff-73647980f94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:44.879225 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdb823-d5b1-48b7-adff-73647980f94a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:44.879570 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdb823-d5b1-48b7-adff-73647980f94a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:44.879912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdb823-d5b1-48b7-adff-73647980f94a None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:44.880689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cdb823-d5b1-48b7-adff-73647980f94a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:44.881062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cdb823-d5b1-48b7-adff-73647980f94a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:44.882619 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cdb823-d5b1-48b7-adff-73647980f94a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:44.893894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 475/948] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 18:04:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:44.895489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4148793-02be-453d-b1f7-68aaad1a8a32 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:45.017077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4148793-02be-453d-b1f7-68aaad1a8a32 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:45.050302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:45.059180 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cdd23-aadc-44fd-b894-b70979c8c7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:45.059867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cdd23-aadc-44fd-b894-b70979c8c7ca None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:45.060144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cdd23-aadc-44fd-b894-b70979c8c7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:45.060411 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cdd23-aadc-44fd-b894-b70979c8c7ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:45.061166 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cdd23-aadc-44fd-b894-b70979c8c7ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:45.062570 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cdd23-aadc-44fd-b894-b70979c8c7ca None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:45.086397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 474/949] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 18:04:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:04:45.611763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0a2a4-20e4-4366-b2d4-ad728b7f2809 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:45.612111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0a2a4-20e4-4366-b2d4-ad728b7f2809 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:45.612324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0a2a4-20e4-4366-b2d4-ad728b7f2809 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:45.613964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 476/950] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:45.619675 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76468d-0cb2-40c1-9cda-25d161fe33b3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:45.620160 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76468d-0cb2-40c1-9cda-25d161fe33b3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:45.620755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76468d-0cb2-40c1-9cda-25d161fe33b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:45.639684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-3d76468d-0cb2-40c1-9cda-25d161fe33b3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:45.641224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3d76468d-0cb2-40c1-9cda-25d161fe33b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:45.728079 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 475/951] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:45.751320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0a2a4-20e4-4366-b2d4-ad728b7f2809 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:45.866117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0a2a4-20e4-4366-b2d4-ad728b7f2809 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:45.905204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01c1d8-5591-4bb9-ac77-11388c2cce01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:45.906310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01c1d8-5591-4bb9-ac77-11388c2cce01 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:45.906738 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01c1d8-5591-4bb9-ac77-11388c2cce01 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:45.907313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01c1d8-5591-4bb9-ac77-11388c2cce01 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:45.925161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01c1d8-5591-4bb9-ac77-11388c2cce01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-bd01c1d8-5591-4bb9-ac77-11388c2cce01 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:04:45.939667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:45.941529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:45.942639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:45.942639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:45.942639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:45.942639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:45.942639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:04:45.942639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:04:45.942639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:04:45.942639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:04:45.942639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:04:45.942639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:04:45.942639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 18:04:45.942639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:45.945314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 477/952] 10.222.0.46 () {56 vars in 1120 bytes} [Thu May 26 18:04:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:45.951595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76468d-0cb2-40c1-9cda-25d161fe33b3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:46.076199 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76468d-0cb2-40c1-9cda-25d161fe33b3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:46.087845 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:46.115055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71add7da-d9dc-4697-a672-22ca2e575026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:46.115737 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71add7da-d9dc-4697-a672-22ca2e575026 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:46.116017 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71add7da-d9dc-4697-a672-22ca2e575026 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:46.116347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71add7da-d9dc-4697-a672-22ca2e575026 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:46.117040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71add7da-d9dc-4697-a672-22ca2e575026 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:46.117352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71add7da-d9dc-4697-a672-22ca2e575026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:46.118783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71add7da-d9dc-4697-a672-22ca2e575026 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:46.128605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 476/953] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 18:04:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:46.133922 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01c1d8-5591-4bb9-ac77-11388c2cce01 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:46.155152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:46.249634 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01c1d8-5591-4bb9-ac77-11388c2cce01 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:46.288365 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9ee2f-86eb-422f-a521-c816b454e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:46.289583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9ee2f-86eb-422f-a521-c816b454e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:46.290206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9ee2f-86eb-422f-a521-c816b454e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:46.290694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9ee2f-86eb-422f-a521-c816b454e9e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:46.303514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9ee2f-86eb-422f-a521-c816b454e9e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:46.305401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9ee2f-86eb-422f-a521-c816b454e9e4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f7f9ee2f-86eb-422f-a521-c816b454e9e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:46.327329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:46.328273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:46.329165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:46.329165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:46.329165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:46.329165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:46.329165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 478/954] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:46.334429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71add7da-d9dc-4697-a672-22ca2e575026 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:46.454831 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71add7da-d9dc-4697-a672-22ca2e575026 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:46.495774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd042a-4e15-4066-a13e-7ca12ff8e9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:46.496541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd042a-4e15-4066-a13e-7ca12ff8e9ca None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:46.496947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd042a-4e15-4066-a13e-7ca12ff8e9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:46.497224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd042a-4e15-4066-a13e-7ca12ff8e9ca None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:46.497978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd042a-4e15-4066-a13e-7ca12ff8e9ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:46.498281 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd042a-4e15-4066-a13e-7ca12ff8e9ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:46.499652 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd042a-4e15-4066-a13e-7ca12ff8e9ca None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:46.517688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 477/955] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:46.518690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9ee2f-86eb-422f-a521-c816b454e9e4 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:46.599106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:46.613342 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654b1b4-3d68-433d-bc32-cd2f97e3144b None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:46.614003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654b1b4-3d68-433d-bc32-cd2f97e3144b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:46.614482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654b1b4-3d68-433d-bc32-cd2f97e3144b None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:46.616488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 478/956] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:46.622927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9510900-60ab-4fee-adf8-6e928261a0ea None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:46.624096 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9510900-60ab-4fee-adf8-6e928261a0ea None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:46.624575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9510900-60ab-4fee-adf8-6e928261a0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:46.632572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9ee2f-86eb-422f-a521-c816b454e9e4 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:46.643490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b9510900-60ab-4fee-adf8-6e928261a0ea None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:46.645001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b9510900-60ab-4fee-adf8-6e928261a0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:46.673009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9899f06-2821-4c4d-ac52-d691dd65029e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:46.673843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9899f06-2821-4c4d-ac52-d691dd65029e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:46.674130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9899f06-2821-4c4d-ac52-d691dd65029e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:46.674425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9899f06-2821-4c4d-ac52-d691dd65029e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:46.675101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:46.683413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9899f06-2821-4c4d-ac52-d691dd65029e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:46.685114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9899f06-2821-4c4d-ac52-d691dd65029e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f9899f06-2821-4c4d-ac52-d691dd65029e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:46.701203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:46.707150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 479/957] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 18:04:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68719bb8-60bb-4713-8065-7ec494183a51 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:46.708121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68719bb8-60bb-4713-8065-7ec494183a51 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:46.708854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68719bb8-60bb-4713-8065-7ec494183a51 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:46.708854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 480/958] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:46.711119 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68719bb8-60bb-4713-8065-7ec494183a51 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:46.733779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 479/959] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:46.733779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54523ff-818d-4f8f-9787-387520dd4ec9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:46.733779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54523ff-818d-4f8f-9787-387520dd4ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:46.733779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54523ff-818d-4f8f-9787-387520dd4ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:46.752307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c54523ff-818d-4f8f-9787-387520dd4ec9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:46.753612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c54523ff-818d-4f8f-9787-387520dd4ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:46.830775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68719bb8-60bb-4713-8065-7ec494183a51 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:46.833869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 480/960] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:46.834948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54523ff-818d-4f8f-9787-387520dd4ec9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:46.864104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:46.870938 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55766878-ec0f-4c27-8cde-8dc947f6c10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:46.871609 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55766878-ec0f-4c27-8cde-8dc947f6c10b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:46.871899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55766878-ec0f-4c27-8cde-8dc947f6c10b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:46.872174 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55766878-ec0f-4c27-8cde-8dc947f6c10b None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:46.873005 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55766878-ec0f-4c27-8cde-8dc947f6c10b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:46.873313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55766878-ec0f-4c27-8cde-8dc947f6c10b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:46.874854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55766878-ec0f-4c27-8cde-8dc947f6c10b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:46.898923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 481/961] 10.222.0.46 () {56 vars in 1149 bytes} [Thu May 26 18:04:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:46.898923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55766878-ec0f-4c27-8cde-8dc947f6c10b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:46.954474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:46.966447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54523ff-818d-4f8f-9787-387520dd4ec9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.002829 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace5364-839e-436c-bfee-db637fbd5c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.003759 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace5364-839e-436c-bfee-db637fbd5c23 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.004110 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace5364-839e-436c-bfee-db637fbd5c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.004451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace5364-839e-436c-bfee-db637fbd5c23 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.005173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace5364-839e-436c-bfee-db637fbd5c23 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.006581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace5364-839e-436c-bfee-db637fbd5c23 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-7ace5364-839e-436c-bfee-db637fbd5c23 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:47.019338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:47.026906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:47.027925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:47.028847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55766878-ec0f-4c27-8cde-8dc947f6c10b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.028946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 481/962] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 18:04:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:47.028946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace5364-839e-436c-bfee-db637fbd5c23 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:47.067078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d3de5-5611-46f6-ab4a-f03d4f60de0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.067747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d3de5-5611-46f6-ab4a-f03d4f60de0d None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.068026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d3de5-5611-46f6-ab4a-f03d4f60de0d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.068299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d3de5-5611-46f6-ab4a-f03d4f60de0d None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.068988 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d3de5-5611-46f6-ab4a-f03d4f60de0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.070658 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d3de5-5611-46f6-ab4a-f03d4f60de0d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.071970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: WARNING keystone.server.flask.common [None req-143d3de5-5611-46f6-ab4a-f03d4f60de0d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 18:04:47.084675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.sql.core [None req-143d3de5-5611-46f6-ab4a-f03d4f60de0d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 18:04:47.084675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 18:04:47.084675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [parameters: {'user_id': '00f1c38060874bde9805eb010ad235a4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 18:04:47.084675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80325) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-143d3de5-5611-46f6-ab4a-f03d4f60de0d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result = self._query(query) May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application conn.query(q) May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result.read() May 26 18:04:47.101907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 18:04:47.103028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application next(self.gen) May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application yield resource May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application next(self.gen) May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.session.rollback() May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.force_reraise() May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise self.value May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application session.commit() May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 18:04:47.104050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._prepare_impl() May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.session.flush() May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._flush(objects) May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application compat.raise_( May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application flush_context.execute() May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rec.execute(self) May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 18:04:47.105828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result = connection._execute_20( May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 18:04:47.106718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result = self._query(query) May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application conn.query(q) May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result.read() May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 18:04:47.107577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [parameters: {'user_id': '00f1c38060874bde9805eb010ad235a4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:47.108451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 26 18:04:47.109336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 26 18:04:47.110282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 26 18:04:47.110282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 26 18:04:47.110282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 18:04:47.110282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 18:04:47.110282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 18:04:47.110282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:47.110282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 482/963] 10.222.0.46 () {60 vars in 1131 bytes} [Thu May 26 18:04:46 2022] POST /identity/v3/users => generated 160 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:47.110282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d3de5-5611-46f6-ab4a-f03d4f60de0d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:47.152279 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace5364-839e-436c-bfee-db637fbd5c23 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.164754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:47.190055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e8101-488d-4d1c-bbb9-22294454368d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.190728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e8101-488d-4d1c-bbb9-22294454368d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.191010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e8101-488d-4d1c-bbb9-22294454368d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.191282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e8101-488d-4d1c-bbb9-22294454368d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.218147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:47.223876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d3de5-5611-46f6-ab4a-f03d4f60de0d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.261499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69851320-f197-4267-bd0a-4092776e00a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.262269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69851320-f197-4267-bd0a-4092776e00a0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.262549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69851320-f197-4267-bd0a-4092776e00a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.262822 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69851320-f197-4267-bd0a-4092776e00a0 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.263529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69851320-f197-4267-bd0a-4092776e00a0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:47.263836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69851320-f197-4267-bd0a-4092776e00a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.265426 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69851320-f197-4267-bd0a-4092776e00a0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.267576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e8101-488d-4d1c-bbb9-22294454368d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.269011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e8101-488d-4d1c-bbb9-22294454368d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.277636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 483/964] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 18:04:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:47.277636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69851320-f197-4267-bd0a-4092776e00a0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:47.390705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 482/965] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:04:47 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:04:47.390705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e8101-488d-4d1c-bbb9-22294454368d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:47.391022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69851320-f197-4267-bd0a-4092776e00a0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.404725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:47.431643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6850a37-4e9a-46f6-861e-f88efa2d1343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.432506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6850a37-4e9a-46f6-861e-f88efa2d1343 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.433370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6850a37-4e9a-46f6-861e-f88efa2d1343 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.433653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6850a37-4e9a-46f6-861e-f88efa2d1343 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.443977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6850a37-4e9a-46f6-861e-f88efa2d1343 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.445900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6850a37-4e9a-46f6-861e-f88efa2d1343 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.453201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-b6850a37-4e9a-46f6-861e-f88efa2d1343 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:04:47.465156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:47.466398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:47.467372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:47.467372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:47.467372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:04:47.467372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:04:47.467372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:04:47.467372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:04:47.467372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:04:47.467372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:04:47.467372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 18:04:47.467372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:47.473222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 484/966] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:04:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:47.473222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6850a37-4e9a-46f6-861e-f88efa2d1343 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:47.502774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e8101-488d-4d1c-bbb9-22294454368d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.539171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e274183-f1b6-4893-bab5-7bdaa52bb618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.539826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e274183-f1b6-4893-bab5-7bdaa52bb618 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.540103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e274183-f1b6-4893-bab5-7bdaa52bb618 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.540364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e274183-f1b6-4893-bab5-7bdaa52bb618 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.541048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e274183-f1b6-4893-bab5-7bdaa52bb618 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:47.541356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e274183-f1b6-4893-bab5-7bdaa52bb618 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.543670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e274183-f1b6-4893-bab5-7bdaa52bb618 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.549314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 483/967] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 18:04:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:47.549976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e274183-f1b6-4893-bab5-7bdaa52bb618 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:47.600989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6850a37-4e9a-46f6-861e-f88efa2d1343 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.637603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc48185-8a3e-4cd5-8caf-5e0b94ff134b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.638685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc48185-8a3e-4cd5-8caf-5e0b94ff134b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.639134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc48185-8a3e-4cd5-8caf-5e0b94ff134b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.639678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc48185-8a3e-4cd5-8caf-5e0b94ff134b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.651214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc48185-8a3e-4cd5-8caf-5e0b94ff134b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.652733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc48185-8a3e-4cd5-8caf-5e0b94ff134b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.661896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e274183-f1b6-4893-bab5-7bdaa52bb618 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-4bc48185-8a3e-4cd5-8caf-5e0b94ff134b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:04:47.672366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:47.673284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:47.674626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:04:47.674626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:04:47.674626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:04:47.674626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:47.674626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 485/968] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:47.674626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc48185-8a3e-4cd5-8caf-5e0b94ff134b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:47.688369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:47.705198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d340d76-281d-40db-ae24-8c052a9d703a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.705986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d340d76-281d-40db-ae24-8c052a9d703a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.706270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d340d76-281d-40db-ae24-8c052a9d703a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.706546 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d340d76-281d-40db-ae24-8c052a9d703a None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.707354 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d340d76-281d-40db-ae24-8c052a9d703a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:47.707662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d340d76-281d-40db-ae24-8c052a9d703a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.709095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d340d76-281d-40db-ae24-8c052a9d703a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.711141 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:47.751565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 484/969] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 18:04:47 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:47.752273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d340d76-281d-40db-ae24-8c052a9d703a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:47.820306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc48185-8a3e-4cd5-8caf-5e0b94ff134b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.868987 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d7bde-7461-4bf7-91b7-51f3ee7d7028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.870785 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d7bde-7461-4bf7-91b7-51f3ee7d7028 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.871067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d7bde-7461-4bf7-91b7-51f3ee7d7028 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.871338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d7bde-7461-4bf7-91b7-51f3ee7d7028 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.872405 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d7bde-7461-4bf7-91b7-51f3ee7d7028 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.874059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d7bde-7461-4bf7-91b7-51f3ee7d7028 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.883619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d340d76-281d-40db-ae24-8c052a9d703a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.905941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 486/970] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 18:04:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:47.939777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5521ab-d998-4e4c-a514-1ef3cce80bbb None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.940131 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5521ab-d998-4e4c-a514-1ef3cce80bbb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.940325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5521ab-d998-4e4c-a514-1ef3cce80bbb None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.942283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 487/971] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:47.949256 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834164c2-dc19-4a75-ad42-694d34fcc64c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.949980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834164c2-dc19-4a75-ad42-694d34fcc64c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.950501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834164c2-dc19-4a75-ad42-694d34fcc64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.950772 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbb77c-6dc7-44ae-b841-22b98c178a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.951520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbb77c-6dc7-44ae-b841-22b98c178a81 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.951802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbb77c-6dc7-44ae-b841-22b98c178a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.952077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbb77c-6dc7-44ae-b841-22b98c178a81 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.952911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbb77c-6dc7-44ae-b841-22b98c178a81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:47.953230 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbb77c-6dc7-44ae-b841-22b98c178a81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.954804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbb77c-6dc7-44ae-b841-22b98c178a81 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.970249 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 485/972] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:47.971581 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-834164c2-dc19-4a75-ad42-694d34fcc64c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:47.973319 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-834164c2-dc19-4a75-ad42-694d34fcc64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:47.977980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbb77c-6dc7-44ae-b841-22b98c178a81 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:48.028151 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:48.093894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 488/973] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:48.112469 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834164c2-dc19-4a75-ad42-694d34fcc64c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:48.119103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbb77c-6dc7-44ae-b841-22b98c178a81 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:48.130994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:48.168535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae6cdc-dedf-4387-b8cb-414db21d9afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:48.169340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae6cdc-dedf-4387-b8cb-414db21d9afb None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:48.169627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae6cdc-dedf-4387-b8cb-414db21d9afb None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:48.169954 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae6cdc-dedf-4387-b8cb-414db21d9afb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:48.177259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae6cdc-dedf-4387-b8cb-414db21d9afb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:48.178739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae6cdc-dedf-4387-b8cb-414db21d9afb None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-98ae6cdc-dedf-4387-b8cb-414db21d9afb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:04:48.196068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:48.197138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:48.202545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:04:48.202545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:04:48.202545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:04:48.202545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:04:48.202545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:04:48.202545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:04:48.202545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:04:48.202545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:48.202545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 486/974] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:48.205140 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae6cdc-dedf-4387-b8cb-414db21d9afb None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:48.228253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834164c2-dc19-4a75-ad42-694d34fcc64c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:48.272949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3312ae-a93d-4e46-8543-df7b41fc2cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:48.272949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3312ae-a93d-4e46-8543-df7b41fc2cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:48.272949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3312ae-a93d-4e46-8543-df7b41fc2cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:48.272949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3312ae-a93d-4e46-8543-df7b41fc2cd6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:48.274184 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3312ae-a93d-4e46-8543-df7b41fc2cd6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:48.275368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3312ae-a93d-4e46-8543-df7b41fc2cd6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9f3312ae-a93d-4e46-8543-df7b41fc2cd6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:48.288818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:48.290063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:48.291022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 489/975] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:04:48.298024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3312ae-a93d-4e46-8543-df7b41fc2cd6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:48.327605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:48.337214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae6cdc-dedf-4387-b8cb-414db21d9afb None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:48.376886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27e6cd-fa0e-4315-9801-f16a70a8a87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:48.377546 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27e6cd-fa0e-4315-9801-f16a70a8a87b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:48.377855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27e6cd-fa0e-4315-9801-f16a70a8a87b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:48.378131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27e6cd-fa0e-4315-9801-f16a70a8a87b None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:48.379032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27e6cd-fa0e-4315-9801-f16a70a8a87b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:48.379343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27e6cd-fa0e-4315-9801-f16a70a8a87b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:48.380896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27e6cd-fa0e-4315-9801-f16a70a8a87b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:48.392375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:48.395949 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 487/976] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:04:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:48.402301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27e6cd-fa0e-4315-9801-f16a70a8a87b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:48.418178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3312ae-a93d-4e46-8543-df7b41fc2cd6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:48.459039 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69334636-276d-4081-b292-64677fd0ef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:48.459710 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69334636-276d-4081-b292-64677fd0ef6e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:48.459989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69334636-276d-4081-b292-64677fd0ef6e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:48.460258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69334636-276d-4081-b292-64677fd0ef6e None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:48.460964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69334636-276d-4081-b292-64677fd0ef6e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:48.461267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69334636-276d-4081-b292-64677fd0ef6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:48.462686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69334636-276d-4081-b292-64677fd0ef6e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:48.468703 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 490/977] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:48.474380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69334636-276d-4081-b292-64677fd0ef6e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:48.527027 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27e6cd-fa0e-4315-9801-f16a70a8a87b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:48.568933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7419e4-1f88-4227-a866-a7718c3ba76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:48.569575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7419e4-1f88-4227-a866-a7718c3ba76c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:48.569895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7419e4-1f88-4227-a866-a7718c3ba76c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:48.570174 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7419e4-1f88-4227-a866-a7718c3ba76c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:48.571050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7419e4-1f88-4227-a866-a7718c3ba76c None admin] RBAC: Adding query filter params (role.id=e9715685617b47c7809732b0a8c61c35, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=90dcc2061d6b4987a1722fab33875b0a) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:48.571365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7419e4-1f88-4227-a866-a7718c3ba76c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:48.572913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7419e4-1f88-4227-a866-a7718c3ba76c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:48.580393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 488/978] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:04:48 2022] GET /identity/v3/role_assignments?user.id=90dcc2061d6b4987a1722fab33875b0a&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=e9715685617b47c7809732b0a8c61c35 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:48.602996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69334636-276d-4081-b292-64677fd0ef6e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:48.642327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6d3e8-f413-4811-b7f5-30f1d18b9ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:48.643334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6d3e8-f413-4811-b7f5-30f1d18b9ecf None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:48.643805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6d3e8-f413-4811-b7f5-30f1d18b9ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:48.644264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6d3e8-f413-4811-b7f5-30f1d18b9ecf None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:48.645211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6d3e8-f413-4811-b7f5-30f1d18b9ecf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:48.645848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6d3e8-f413-4811-b7f5-30f1d18b9ecf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:48.647486 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6d3e8-f413-4811-b7f5-30f1d18b9ecf None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:48.654008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 491/979] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:48 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:50.225556 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9173d-a053-4e04-bc3f-ea27725d3312 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:50.226268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9173d-a053-4e04-bc3f-ea27725d3312 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:50.226268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9173d-a053-4e04-bc3f-ea27725d3312 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:50.227671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 489/980] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:50.233115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7a694-aad4-4db6-9fd4-5a60aeeb0ff7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:50.233639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7a694-aad4-4db6-9fd4-5a60aeeb0ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:50.233639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7a694-aad4-4db6-9fd4-5a60aeeb0ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:50.239626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:50.253429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a8c7a694-aad4-4db6-9fd4-5a60aeeb0ff7 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:50.254867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a8c7a694-aad4-4db6-9fd4-5a60aeeb0ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:50.345777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 492/981] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:50.383770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9173d-a053-4e04-bc3f-ea27725d3312 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:50.414071 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:50.517841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9173d-a053-4e04-bc3f-ea27725d3312 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:50.561314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959a396-5a80-4936-968a-9e699203443b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:50.562160 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959a396-5a80-4936-968a-9e699203443b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:50.562432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959a396-5a80-4936-968a-9e699203443b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:50.562697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959a396-5a80-4936-968a-9e699203443b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:50.563348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959a396-5a80-4936-968a-9e699203443b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:50.564710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959a396-5a80-4936-968a-9e699203443b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-5959a396-5a80-4936-968a-9e699203443b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:50.577310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:50.578584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:50.579503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 490/982] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 18:04:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:50.586487 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7a694-aad4-4db6-9fd4-5a60aeeb0ff7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:50.713526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7a694-aad4-4db6-9fd4-5a60aeeb0ff7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:50.757164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740615e0-e007-4213-a2e5-1b6295f1365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:50.758380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740615e0-e007-4213-a2e5-1b6295f1365f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:50.758380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740615e0-e007-4213-a2e5-1b6295f1365f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:50.758380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740615e0-e007-4213-a2e5-1b6295f1365f None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:50.759110 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740615e0-e007-4213-a2e5-1b6295f1365f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:50.759412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740615e0-e007-4213-a2e5-1b6295f1365f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:50.761442 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740615e0-e007-4213-a2e5-1b6295f1365f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:50.763292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:50.782559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 493/983] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 18:04:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:50.783974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959a396-5a80-4936-968a-9e699203443b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:50.880441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:50.902563 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959a396-5a80-4936-968a-9e699203443b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:50.905343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d262b7c-4ace-4f8f-b4b4-7dcac214e6c3 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:50.905622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d262b7c-4ace-4f8f-b4b4-7dcac214e6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:50.906044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d262b7c-4ace-4f8f-b4b4-7dcac214e6c3 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:50.907547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 494/984] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:50.913093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8d37c-8413-4cc4-b9e7-6ecf38a0d6c6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:50.913381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8d37c-8413-4cc4-b9e7-6ecf38a0d6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:50.913652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8d37c-8413-4cc4-b9e7-6ecf38a0d6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:50.931948 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d2d8d37c-8413-4cc4-b9e7-6ecf38a0d6c6 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:50.934103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d2d8d37c-8413-4cc4-b9e7-6ecf38a0d6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:50.947580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d076d9-af86-4056-8e79-4afc3a6741ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:50.948277 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d076d9-af86-4056-8e79-4afc3a6741ea None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:50.949096 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d076d9-af86-4056-8e79-4afc3a6741ea None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:50.949361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d076d9-af86-4056-8e79-4afc3a6741ea None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:50.950099 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d076d9-af86-4056-8e79-4afc3a6741ea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:50.950410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d076d9-af86-4056-8e79-4afc3a6741ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:50.951786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d076d9-af86-4056-8e79-4afc3a6741ea None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:50.957660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 491/985] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 18:04:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:50.959054 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd62e53-15fd-4e98-9d47-b2a3b7340041 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:50.959344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd62e53-15fd-4e98-9d47-b2a3b7340041 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:50.959615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd62e53-15fd-4e98-9d47-b2a3b7340041 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:50.960853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 492/986] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:50.963616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd62e53-15fd-4e98-9d47-b2a3b7340041 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:51.029878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 495/987] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:51.029878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443da4d9-76e9-4212-9ab3-b7e5f2347f98 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:51.029878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443da4d9-76e9-4212-9ab3-b7e5f2347f98 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:51.030317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443da4d9-76e9-4212-9ab3-b7e5f2347f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:51.056972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-443da4d9-76e9-4212-9ab3-b7e5f2347f98 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:51.058383 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-443da4d9-76e9-4212-9ab3-b7e5f2347f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:51.089514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd62e53-15fd-4e98-9d47-b2a3b7340041 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:51.129264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e589ee9-71bc-4d45-825f-2baeb98d27fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:51.130329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e589ee9-71bc-4d45-825f-2baeb98d27fb None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:51.130610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e589ee9-71bc-4d45-825f-2baeb98d27fb None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:51.130885 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e589ee9-71bc-4d45-825f-2baeb98d27fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:51.131684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e589ee9-71bc-4d45-825f-2baeb98d27fb None admin] RBAC: Adding query filter params (interface=admin, service_id=90054f6df98f4562adf8441146fd1248, region_id=RegionOne) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:51.131999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e589ee9-71bc-4d45-825f-2baeb98d27fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:51.133369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e589ee9-71bc-4d45-825f-2baeb98d27fb None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:51.133722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 496/988] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:51.134679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443da4d9-76e9-4212-9ab3-b7e5f2347f98 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:51.139864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 493/989] 10.222.0.46 () {56 vars in 1249 bytes} [Thu May 26 18:04:50 2022] GET /identity/v3/endpoints?service_id=90054f6df98f4562adf8441146fd1248&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:51.148550 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e589ee9-71bc-4d45-825f-2baeb98d27fb None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:51.168660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:51.223496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:51.281753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443da4d9-76e9-4212-9ab3-b7e5f2347f98 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:51.291647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e589ee9-71bc-4d45-825f-2baeb98d27fb None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:51.322314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e801b-0f63-42b4-887e-a842f3b267ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:51.323115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e801b-0f63-42b4-887e-a842f3b267ab None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:51.323421 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e801b-0f63-42b4-887e-a842f3b267ab None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:51.323721 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e801b-0f63-42b4-887e-a842f3b267ab None admin] PATH_INFO: `/v3/services/image` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:51.324354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e801b-0f63-42b4-887e-a842f3b267ab None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:51.328110 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e801b-0f63-42b4-887e-a842f3b267ab None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:51.329682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99b852-c78b-4af2-84f4-b2a6d46b78c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:51.330390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99b852-c78b-4af2-84f4-b2a6d46b78c4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:51.330672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99b852-c78b-4af2-84f4-b2a6d46b78c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:51.330941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99b852-c78b-4af2-84f4-b2a6d46b78c4 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:51.331616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f99b852-c78b-4af2-84f4-b2a6d46b78c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:51.332978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f99b852-c78b-4af2-84f4-b2a6d46b78c4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-bf7e801b-0f63-42b4-887e-a842f3b267ab None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:51.337921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:51.338961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:51.339908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 497/990] 10.222.0.46 () {56 vars in 1105 bytes} [Thu May 26 18:04:51 2022] GET /identity/v3/services/image => generated 86 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:51.340711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e801b-0f63-42b4-887e-a842f3b267ab None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:51.341440 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 494/991] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 18:04:51 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:04:51.346798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99b852-c78b-4af2-84f4-b2a6d46b78c4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:51.453867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:51.485663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99b852-c78b-4af2-84f4-b2a6d46b78c4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:51.486518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e801b-0f63-42b4-887e-a842f3b267ab None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:51.521277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:51.527328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecef9a-9d98-47a1-a889-5d42bf755580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:51.527328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecef9a-9d98-47a1-a889-5d42bf755580 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:51.527328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecef9a-9d98-47a1-a889-5d42bf755580 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:51.527328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecef9a-9d98-47a1-a889-5d42bf755580 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:51.530028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecef9a-9d98-47a1-a889-5d42bf755580 None admin] RBAC: Adding query filter params (name=image) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:51.530028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecef9a-9d98-47a1-a889-5d42bf755580 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:51.530028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecef9a-9d98-47a1-a889-5d42bf755580 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:51.535473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 495/992] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:51.541333 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecef9a-9d98-47a1-a889-5d42bf755580 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:51.545511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec71932-0cbf-48f4-889e-fd6347e51b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:51.546340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec71932-0cbf-48f4-889e-fd6347e51b93 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:51.546622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec71932-0cbf-48f4-889e-fd6347e51b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:51.546892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec71932-0cbf-48f4-889e-fd6347e51b93 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:51.548497 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec71932-0cbf-48f4-889e-fd6347e51b93 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:51.549781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec71932-0cbf-48f4-889e-fd6347e51b93 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:51.566392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 498/993] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:04:51 2022] POST /identity/v3/services => generated 242 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:51.688678 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecef9a-9d98-47a1-a889-5d42bf755580 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:51.737618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbda85-2e1c-4392-a8f7-6df632230324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:51.738336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbda85-2e1c-4392-a8f7-6df632230324 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:51.738618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbda85-2e1c-4392-a8f7-6df632230324 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:51.738905 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbda85-2e1c-4392-a8f7-6df632230324 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:51.739614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbda85-2e1c-4392-a8f7-6df632230324 None admin] RBAC: Adding query filter params (type=image) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:51.739924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbda85-2e1c-4392-a8f7-6df632230324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:51.741386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbda85-2e1c-4392-a8f7-6df632230324 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:51.743015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:51.757766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 496/994] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:53.754905 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505d753-8d39-4d43-aa87-3fd3cf1ccf4b None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.755456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505d753-8d39-4d43-aa87-3fd3cf1ccf4b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.755727 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505d753-8d39-4d43-aa87-3fd3cf1ccf4b None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.757271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 499/995] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:53.764601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3758430-8d19-43e9-ac14-cc57daa3c0d9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.765199 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3758430-8d19-43e9-ac14-cc57daa3c0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.765668 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3758430-8d19-43e9-ac14-cc57daa3c0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.784103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c3758430-8d19-43e9-ac14-cc57daa3c0d9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:53.785349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c3758430-8d19-43e9-ac14-cc57daa3c0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:53.864526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 497/996] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:53.882348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7906f-7848-4316-a3ad-f8f99f5b363b None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.882699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7906f-7848-4316-a3ad-f8f99f5b363b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.882909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7906f-7848-4316-a3ad-f8f99f5b363b None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.884434 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 500/997] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:53.888350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3758430-8d19-43e9-ac14-cc57daa3c0d9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:53.890457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1dab1-dd5f-4018-8913-9a7657d23ee7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.890746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1dab1-dd5f-4018-8913-9a7657d23ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.891027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1dab1-dd5f-4018-8913-9a7657d23ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.908901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3ef1dab1-dd5f-4018-8913-9a7657d23ee7 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:53.910123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3ef1dab1-dd5f-4018-8913-9a7657d23ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.016443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3758430-8d19-43e9-ac14-cc57daa3c0d9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.016642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 501/998] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:54.034382 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5b91e-e7d3-45f1-93cc-0d54299470f9 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.034706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5b91e-e7d3-45f1-93cc-0d54299470f9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.034942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5b91e-e7d3-45f1-93cc-0d54299470f9 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.036601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 502/999] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:54.042916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f12570-be4b-43e3-9426-2aca07e58a4a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.043204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f12570-be4b-43e3-9426-2aca07e58a4a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.043463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f12570-be4b-43e3-9426-2aca07e58a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.057028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:54.060359 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-56f12570-be4b-43e3-9426-2aca07e58a4a None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:54.061604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-56f12570-be4b-43e3-9426-2aca07e58a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.066303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9dd34-ec25-4eb7-a6c0-b0d3759dfe49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.067119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9dd34-ec25-4eb7-a6c0-b0d3759dfe49 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.074708 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9dd34-ec25-4eb7-a6c0-b0d3759dfe49 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.074708 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9dd34-ec25-4eb7-a6c0-b0d3759dfe49 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.074708 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9dd34-ec25-4eb7-a6c0-b0d3759dfe49 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.074708 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9dd34-ec25-4eb7-a6c0-b0d3759dfe49 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-22e9dd34-ec25-4eb7-a6c0-b0d3759dfe49 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:54.082334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:54.083593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:54.086684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 498/1000] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 18:04:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:54.087485 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9dd34-ec25-4eb7-a6c0-b0d3759dfe49 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.145570 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 503/1001] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:54.146470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f12570-be4b-43e3-9426-2aca07e58a4a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.199519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:54.203568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9dd34-ec25-4eb7-a6c0-b0d3759dfe49 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.240112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74b20a-1483-4a66-9159-bc2df15dda05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.241014 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74b20a-1483-4a66-9159-bc2df15dda05 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.241385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74b20a-1483-4a66-9159-bc2df15dda05 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.241750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74b20a-1483-4a66-9159-bc2df15dda05 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.242407 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74b20a-1483-4a66-9159-bc2df15dda05 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.243776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74b20a-1483-4a66-9159-bc2df15dda05 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-cb74b20a-1483-4a66-9159-bc2df15dda05 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:54.256792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:54.257936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:54.259087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 499/1002] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:54.260186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74b20a-1483-4a66-9159-bc2df15dda05 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.276031 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f12570-be4b-43e3-9426-2aca07e58a4a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.318518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945f588-d027-429b-91d6-f5ac49720852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.319176 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945f588-d027-429b-91d6-f5ac49720852 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.319453 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945f588-d027-429b-91d6-f5ac49720852 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.319719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945f588-d027-429b-91d6-f5ac49720852 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.320421 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945f588-d027-429b-91d6-f5ac49720852 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:54.320716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945f588-d027-429b-91d6-f5ac49720852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.322254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945f588-d027-429b-91d6-f5ac49720852 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.335144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 504/1003] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 18:04:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:54.336014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945f588-d027-429b-91d6-f5ac49720852 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.349473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:54.401073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74b20a-1483-4a66-9159-bc2df15dda05 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.437744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff005cf-d110-4e0f-8b5e-c287da9df388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.438431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff005cf-d110-4e0f-8b5e-c287da9df388 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.438707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff005cf-d110-4e0f-8b5e-c287da9df388 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.438978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff005cf-d110-4e0f-8b5e-c287da9df388 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.439704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff005cf-d110-4e0f-8b5e-c287da9df388 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.441093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff005cf-d110-4e0f-8b5e-c287da9df388 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.453313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945f588-d027-429b-91d6-f5ac49720852 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.469764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 500/1004] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:04:54 2022] POST /identity/v3/services => generated 242 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:54.469764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff005cf-d110-4e0f-8b5e-c287da9df388 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.470068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:54.518445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a64f0-0a8c-4fcf-b4f2-2fbbe62a79c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.519143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a64f0-0a8c-4fcf-b4f2-2fbbe62a79c3 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.519464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a64f0-0a8c-4fcf-b4f2-2fbbe62a79c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.519737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a64f0-0a8c-4fcf-b4f2-2fbbe62a79c3 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.520523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a64f0-0a8c-4fcf-b4f2-2fbbe62a79c3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:54.520962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a64f0-0a8c-4fcf-b4f2-2fbbe62a79c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.523047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a64f0-0a8c-4fcf-b4f2-2fbbe62a79c3 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.533642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 505/1005] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:54.535518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a64f0-0a8c-4fcf-b4f2-2fbbe62a79c3 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.635046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff005cf-d110-4e0f-8b5e-c287da9df388 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.662771 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:54.694896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048a536-1b05-4943-8520-89b5d06825b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.696130 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048a536-1b05-4943-8520-89b5d06825b2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.696241 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048a536-1b05-4943-8520-89b5d06825b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.696584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048a536-1b05-4943-8520-89b5d06825b2 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.697416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048a536-1b05-4943-8520-89b5d06825b2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:54.697965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048a536-1b05-4943-8520-89b5d06825b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.700647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048a536-1b05-4943-8520-89b5d06825b2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.708250 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a64f0-0a8c-4fcf-b4f2-2fbbe62a79c3 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.708954 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 501/1006] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 18:04:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:54.715484 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048a536-1b05-4943-8520-89b5d06825b2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.760534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccba683-acdb-4455-a330-9f039fcb743a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.762024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccba683-acdb-4455-a330-9f039fcb743a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.762449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccba683-acdb-4455-a330-9f039fcb743a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.763063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccba683-acdb-4455-a330-9f039fcb743a None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.763854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccba683-acdb-4455-a330-9f039fcb743a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:54.764385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccba683-acdb-4455-a330-9f039fcb743a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.766297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccba683-acdb-4455-a330-9f039fcb743a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.775619 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 506/1007] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:54.781455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccba683-acdb-4455-a330-9f039fcb743a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.829019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:54.866440 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048a536-1b05-4943-8520-89b5d06825b2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.917240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccba683-acdb-4455-a330-9f039fcb743a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.922320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10366cc-eee2-495a-aa6f-ed8b608f1850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.923033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10366cc-eee2-495a-aa6f-ed8b608f1850 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.923289 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10366cc-eee2-495a-aa6f-ed8b608f1850 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.923557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10366cc-eee2-495a-aa6f-ed8b608f1850 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.924263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10366cc-eee2-495a-aa6f-ed8b608f1850 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.925630 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10366cc-eee2-495a-aa6f-ed8b608f1850 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.941681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 502/1008] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 18:04:54 2022] POST /identity/v3/endpoints => generated 325 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:04:54.968610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f75703-666b-42f7-8948-37f53617cebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.969466 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f75703-666b-42f7-8948-37f53617cebc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.969866 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f75703-666b-42f7-8948-37f53617cebc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.970328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f75703-666b-42f7-8948-37f53617cebc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.976562 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f75703-666b-42f7-8948-37f53617cebc None admin] RBAC: Adding query filter params (interface=public, service_id=387cff3d2ac8413bb753de4898f64f59, region_id=RegionOne) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:54.976880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f75703-666b-42f7-8948-37f53617cebc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.978561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f75703-666b-42f7-8948-37f53617cebc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.986636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 507/1009] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 18:04:54 2022] GET /identity/v3/endpoints?service_id=387cff3d2ac8413bb753de4898f64f59&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:54.992407 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10366cc-eee2-495a-aa6f-ed8b608f1850 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:55.062760 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:55.141401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10366cc-eee2-495a-aa6f-ed8b608f1850 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:55.179109 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937a154-0eeb-4427-a955-eaa6b46ea0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:55.180139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937a154-0eeb-4427-a955-eaa6b46ea0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:55.180419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937a154-0eeb-4427-a955-eaa6b46ea0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:55.180691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937a154-0eeb-4427-a955-eaa6b46ea0b9 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:55.181384 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937a154-0eeb-4427-a955-eaa6b46ea0b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:55.183438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937a154-0eeb-4427-a955-eaa6b46ea0b9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:55.191607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 503/1010] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 18:04:54 2022] GET /identity/v3/services => generated 1957 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:56.782558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747452d1-fc38-48f8-af2c-69c5e4a2eb0f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:56.783187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747452d1-fc38-48f8-af2c-69c5e4a2eb0f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:56.783187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747452d1-fc38-48f8-af2c-69c5e4a2eb0f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:56.785672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 508/1011] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:56.790779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adf64b-c60a-436e-b44c-a7fa6abf3d07 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:56.791314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adf64b-c60a-436e-b44c-a7fa6abf3d07 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:56.791406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adf64b-c60a-436e-b44c-a7fa6abf3d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:56.812013 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-51adf64b-c60a-436e-b44c-a7fa6abf3d07 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:56.813225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-51adf64b-c60a-436e-b44c-a7fa6abf3d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:56.899430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 504/1012] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:56 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:56.922446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747452d1-fc38-48f8-af2c-69c5e4a2eb0f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.076974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:57.091909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747452d1-fc38-48f8-af2c-69c5e4a2eb0f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:57.142209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551f645-9c93-4d03-8c62-29daa60f895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:57.143813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551f645-9c93-4d03-8c62-29daa60f895e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.144256 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551f645-9c93-4d03-8c62-29daa60f895e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.144695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551f645-9c93-4d03-8c62-29daa60f895e None admin] PATH_INFO: `/v3/services/image` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.145545 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551f645-9c93-4d03-8c62-29daa60f895e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:57.147239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551f645-9c93-4d03-8c62-29daa60f895e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-1551f645-9c93-4d03-8c62-29daa60f895e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:57.161253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:57.162812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 18:04:57.163740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:04:57.165950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 509/1013] 10.222.0.46 () {56 vars in 1105 bytes} [Thu May 26 18:04:56 2022] GET /identity/v3/services/image => generated 86 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:57.171490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51adf64b-c60a-436e-b44c-a7fa6abf3d07 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.223647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:57.300999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51adf64b-c60a-436e-b44c-a7fa6abf3d07 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:57.338819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b130a-eb9d-4064-9463-bd22722d1ebd None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.339051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b130a-eb9d-4064-9463-bd22722d1ebd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.339797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b130a-eb9d-4064-9463-bd22722d1ebd None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.341534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 510/1014] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:57.347889 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8048c7-531b-4698-bad6-3c5b3581ffa5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.348261 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8048c7-531b-4698-bad6-3c5b3581ffa5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.348510 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8048c7-531b-4698-bad6-3c5b3581ffa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.357169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f9973-d49e-4905-9743-a11d0136afae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:57.358336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f9973-d49e-4905-9743-a11d0136afae None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.358790 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f9973-d49e-4905-9743-a11d0136afae None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.359250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f9973-d49e-4905-9743-a11d0136afae None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.360481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f9973-d49e-4905-9743-a11d0136afae None admin] RBAC: Adding query filter params (name=image) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:57.360931 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f9973-d49e-4905-9743-a11d0136afae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:57.363448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f9973-d49e-4905-9743-a11d0136afae None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:57.368041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4b8048c7-531b-4698-bad6-3c5b3581ffa5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:57.369337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4b8048c7-531b-4698-bad6-3c5b3581ffa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:57.371618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 505/1015] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:57.377442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f9973-d49e-4905-9743-a11d0136afae None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.481532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 511/1016] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:57 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:04:57.504131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:57.506063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8048c7-531b-4698-bad6-3c5b3581ffa5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.510215 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f9973-d49e-4905-9743-a11d0136afae None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:57.569163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95db8f-fae4-4c21-b85b-895f70c6a3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:57.569867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95db8f-fae4-4c21-b85b-895f70c6a3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.570152 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95db8f-fae4-4c21-b85b-895f70c6a3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.570432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95db8f-fae4-4c21-b85b-895f70c6a3a1 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.571139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95db8f-fae4-4c21-b85b-895f70c6a3a1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:57.571445 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95db8f-fae4-4c21-b85b-895f70c6a3a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:57.572824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95db8f-fae4-4c21-b85b-895f70c6a3a1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:57.578102 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 506/1017] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:57.579389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf449e8-835b-4024-8dcc-8d74dcd17411 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.579674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf449e8-835b-4024-8dcc-8d74dcd17411 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.579949 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf449e8-835b-4024-8dcc-8d74dcd17411 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.581258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 507/1018] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:57.586304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf449e8-835b-4024-8dcc-8d74dcd17411 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.607994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:57.625078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8048c7-531b-4698-bad6-3c5b3581ffa5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:57.674157 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c1880-882a-469b-a60c-5cf87487e6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:57.674956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c1880-882a-469b-a60c-5cf87487e6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.675237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c1880-882a-469b-a60c-5cf87487e6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.675508 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c1880-882a-469b-a60c-5cf87487e6b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.676762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c1880-882a-469b-a60c-5cf87487e6b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:57.678263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c1880-882a-469b-a60c-5cf87487e6b6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:57.680244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 512/1019] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:04:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:57.681517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64398b89-6212-44b0-90c1-4b8b439e41f3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.681828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64398b89-6212-44b0-90c1-4b8b439e41f3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.682097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64398b89-6212-44b0-90c1-4b8b439e41f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.699369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-64398b89-6212-44b0-90c1-4b8b439e41f3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:57.700655 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-64398b89-6212-44b0-90c1-4b8b439e41f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:57.721522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf449e8-835b-4024-8dcc-8d74dcd17411 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:57.764926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf232e47-7419-4eba-bb62-260569d89e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:57.765591 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf232e47-7419-4eba-bb62-260569d89e6f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.765904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf232e47-7419-4eba-bb62-260569d89e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.766179 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf232e47-7419-4eba-bb62-260569d89e6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.767097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf232e47-7419-4eba-bb62-260569d89e6f None admin] RBAC: Adding query filter params (interface=public, service_id=75fcc685bcb64990b8111b4d705d822e, region_id=RegionOne) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:57.767281 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf232e47-7419-4eba-bb62-260569d89e6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:57.768661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf232e47-7419-4eba-bb62-260569d89e6f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:57.775313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 508/1020] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 18:04:57 2022] GET /identity/v3/endpoints?service_id=75fcc685bcb64990b8111b4d705d822e&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:57.776123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf232e47-7419-4eba-bb62-260569d89e6f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.782092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 513/1021] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:04:57 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:04:57.782952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64398b89-6212-44b0-90c1-4b8b439e41f3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.812076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:57.881791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:57.891247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf232e47-7419-4eba-bb62-260569d89e6f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:57.901215 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64398b89-6212-44b0-90c1-4b8b439e41f3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:57.929081 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdaf188-d12c-449c-aca8-0e126e4e27cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:57.929795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaf188-d12c-449c-aca8-0e126e4e27cc None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.930048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaf188-d12c-449c-aca8-0e126e4e27cc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.930318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaf188-d12c-449c-aca8-0e126e4e27cc None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.931028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdaf188-d12c-449c-aca8-0e126e4e27cc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:57.932467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdaf188-d12c-449c-aca8-0e126e4e27cc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:57.945092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c6bb2-bfa6-4cfd-abfe-738cee30f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:57.946447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c6bb2-bfa6-4cfd-abfe-738cee30f7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.946913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c6bb2-bfa6-4cfd-abfe-738cee30f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.947369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c6bb2-bfa6-4cfd-abfe-738cee30f7d4 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.949111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c6bb2-bfa6-4cfd-abfe-738cee30f7d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:57.951552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c6bb2-bfa6-4cfd-abfe-738cee30f7d4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:57.960427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 509/1022] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:04:57 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:57.961259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdaf188-d12c-449c-aca8-0e126e4e27cc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.977760 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 514/1023] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 18:04:57 2022] GET /identity/v3/services => generated 1957 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:58.084695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:58.121120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdaf188-d12c-449c-aca8-0e126e4e27cc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.165604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119018aa-9a75-4539-beb9-eaf306049946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.166452 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119018aa-9a75-4539-beb9-eaf306049946 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.166735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119018aa-9a75-4539-beb9-eaf306049946 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.167286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119018aa-9a75-4539-beb9-eaf306049946 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.167903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119018aa-9a75-4539-beb9-eaf306049946 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.169406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119018aa-9a75-4539-beb9-eaf306049946 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-119018aa-9a75-4539-beb9-eaf306049946 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:04:58.182626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:04:58.183719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:04:58.184767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 510/1024] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:04:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:04:58.190502 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c6bb2-bfa6-4cfd-abfe-738cee30f7d4 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:58.337592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c6bb2-bfa6-4cfd-abfe-738cee30f7d4 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.382146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcdb72-0081-4cf3-8530-ec909184c432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.382818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcdb72-0081-4cf3-8530-ec909184c432 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.383096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcdb72-0081-4cf3-8530-ec909184c432 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.383374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcdb72-0081-4cf3-8530-ec909184c432 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.384135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcdb72-0081-4cf3-8530-ec909184c432 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:58.384444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcdb72-0081-4cf3-8530-ec909184c432 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.385907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcdb72-0081-4cf3-8530-ec909184c432 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:58.392320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 515/1025] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:04:58.398426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119018aa-9a75-4539-beb9-eaf306049946 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:58.520284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119018aa-9a75-4539-beb9-eaf306049946 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.568987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd591d-7155-469e-a543-6ef7118467e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.569668 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd591d-7155-469e-a543-6ef7118467e3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.569989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd591d-7155-469e-a543-6ef7118467e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.570258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd591d-7155-469e-a543-6ef7118467e3 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.570946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd591d-7155-469e-a543-6ef7118467e3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:58.571249 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd591d-7155-469e-a543-6ef7118467e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.572629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd591d-7155-469e-a543-6ef7118467e3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:58.582157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 511/1026] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 18:04:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:58.587547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcdb72-0081-4cf3-8530-ec909184c432 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:58.600135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:58.706111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcdb72-0081-4cf3-8530-ec909184c432 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.741910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb5e63-cb3b-49a7-b2b1-34ec4f576caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.742572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb5e63-cb3b-49a7-b2b1-34ec4f576caa None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.743082 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb5e63-cb3b-49a7-b2b1-34ec4f576caa None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.743490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb5e63-cb3b-49a7-b2b1-34ec4f576caa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.744261 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb5e63-cb3b-49a7-b2b1-34ec4f576caa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.746281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb5e63-cb3b-49a7-b2b1-34ec4f576caa None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:58.773702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 516/1027] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 18:04:58 2022] POST /identity/v3/endpoints => generated 334 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:00.346541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c37a6e-68b5-4039-b5fc-9edbc2a0d613 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.347051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c37a6e-68b5-4039-b5fc-9edbc2a0d613 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.347155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c37a6e-68b5-4039-b5fc-9edbc2a0d613 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.348597 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 512/1028] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:00.353980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd83ac7-58a7-447a-9936-4edb1616276f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.354939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd83ac7-58a7-447a-9936-4edb1616276f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.355137 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd83ac7-58a7-447a-9936-4edb1616276f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.372717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-0fd83ac7-58a7-447a-9936-4edb1616276f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:00.373952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0fd83ac7-58a7-447a-9936-4edb1616276f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:00.376929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf89b3-05ee-4284-83de-36776545b31c None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.377223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf89b3-05ee-4284-83de-36776545b31c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.377498 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf89b3-05ee-4284-83de-36776545b31c None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.378894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 513/1029] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:00.383665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d78280-aa05-40e8-975f-79f4aaa21b25 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.383950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d78280-aa05-40e8-975f-79f4aaa21b25 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.384224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d78280-aa05-40e8-975f-79f4aaa21b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.396059 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:00.402277 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-87d78280-aa05-40e8-975f-79f4aaa21b25 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:00.403459 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-87d78280-aa05-40e8-975f-79f4aaa21b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:00.448988 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:00.465568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 517/1030] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:00.488234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 514/1031] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:05:00.489963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd83ac7-58a7-447a-9936-4edb1616276f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:00.512292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d78280-aa05-40e8-975f-79f4aaa21b25 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:00.611855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd83ac7-58a7-447a-9936-4edb1616276f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:00.618732 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d78280-aa05-40e8-975f-79f4aaa21b25 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:00.648941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5957754-5b5d-4a33-a3f1-2d311f32c536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:00.649747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5957754-5b5d-4a33-a3f1-2d311f32c536 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.650029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5957754-5b5d-4a33-a3f1-2d311f32c536 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.650297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5957754-5b5d-4a33-a3f1-2d311f32c536 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.651738 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5957754-5b5d-4a33-a3f1-2d311f32c536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:00.653496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5957754-5b5d-4a33-a3f1-2d311f32c536 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:00.655287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 518/1032] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:00.660541 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5957754-5b5d-4a33-a3f1-2d311f32c536 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:00.661295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e086e6-d666-4d60-a1df-647045308923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:00.662541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e086e6-d666-4d60-a1df-647045308923 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.662987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e086e6-d666-4d60-a1df-647045308923 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.663757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e086e6-d666-4d60-a1df-647045308923 None admin] PATH_INFO: `/v3/services/image` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.664780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e086e6-d666-4d60-a1df-647045308923 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:00.667562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e086e6-d666-4d60-a1df-647045308923 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:00.671036 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a4e086e6-d666-4d60-a1df-647045308923 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:00.685326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:00.686422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 18:05:00.687268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:00.693788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 515/1033] 10.222.0.46 () {56 vars in 1105 bytes} [Thu May 26 18:05:00 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:00.696382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e086e6-d666-4d60-a1df-647045308923 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:00.732648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:00.798130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5957754-5b5d-4a33-a3f1-2d311f32c536 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:00.819283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e086e6-d666-4d60-a1df-647045308923 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:00.834189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2a68b-9ca4-41b8-92fa-f700612a401b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:00.834855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2a68b-9ca4-41b8-92fa-f700612a401b None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.835137 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2a68b-9ca4-41b8-92fa-f700612a401b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.835401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2a68b-9ca4-41b8-92fa-f700612a401b None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.836097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2a68b-9ca4-41b8-92fa-f700612a401b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:00.837534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2a68b-9ca4-41b8-92fa-f700612a401b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:00.854281 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc997f4d-3255-4008-972c-5d5399b261b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:00.855007 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc997f4d-3255-4008-972c-5d5399b261b1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.855287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc997f4d-3255-4008-972c-5d5399b261b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.855557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc997f4d-3255-4008-972c-5d5399b261b1 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.856304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc997f4d-3255-4008-972c-5d5399b261b1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:00.856552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc997f4d-3255-4008-972c-5d5399b261b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:00.857909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc997f4d-3255-4008-972c-5d5399b261b1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:00.859559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 519/1034] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:05:00 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:05:00.871237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 516/1035] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:00.877265 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2a68b-9ca4-41b8-92fa-f700612a401b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:00.990106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2a68b-9ca4-41b8-92fa-f700612a401b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:01.012725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:01.038491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e66e55-3903-4902-90ea-d890bc99493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:01.039164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e66e55-3903-4902-90ea-d890bc99493c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:01.039448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e66e55-3903-4902-90ea-d890bc99493c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:01.039722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e66e55-3903-4902-90ea-d890bc99493c None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:01.040433 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e66e55-3903-4902-90ea-d890bc99493c None admin] RBAC: Adding query filter params (type=image) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:01.040741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e66e55-3903-4902-90ea-d890bc99493c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:01.042212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e66e55-3903-4902-90ea-d890bc99493c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:01.047980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 520/1036] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:01.053488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc997f4d-3255-4008-972c-5d5399b261b1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:01.121533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:01.163169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc997f4d-3255-4008-972c-5d5399b261b1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:01.196435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9d791-38dd-4c7a-867d-7030233c3f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:01.197171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9d791-38dd-4c7a-867d-7030233c3f6f None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:01.197449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9d791-38dd-4c7a-867d-7030233c3f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:01.197721 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9d791-38dd-4c7a-867d-7030233c3f6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:01.198873 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac0c99-a7ed-4f09-8538-ca2b0997bb6c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:01.199025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9d791-38dd-4c7a-867d-7030233c3f6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:01.199171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac0c99-a7ed-4f09-8538-ca2b0997bb6c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:01.199443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac0c99-a7ed-4f09-8538-ca2b0997bb6c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:01.200362 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9d791-38dd-4c7a-867d-7030233c3f6f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:01.201135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 521/1037] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:01.206087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ccd6-10a4-430e-b55c-5c52bedab59d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:01.206384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ccd6-10a4-430e-b55c-5c52bedab59d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:01.206738 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ccd6-10a4-430e-b55c-5c52bedab59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:01.222213 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 517/1038] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 18:05:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:01.230641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-99f1ccd6-10a4-430e-b55c-5c52bedab59d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:01.232168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-99f1ccd6-10a4-430e-b55c-5c52bedab59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:01.353780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 522/1039] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:05:01.377523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9d791-38dd-4c7a-867d-7030233c3f6f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:01.528339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9d791-38dd-4c7a-867d-7030233c3f6f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:01.550298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:01.580203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853fcddb-2025-4924-811f-78583a411dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:01.581024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853fcddb-2025-4924-811f-78583a411dff None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:01.581305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853fcddb-2025-4924-811f-78583a411dff None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:01.581576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853fcddb-2025-4924-811f-78583a411dff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:01.593278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853fcddb-2025-4924-811f-78583a411dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-853fcddb-2025-4924-811f-78583a411dff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:05:01.601794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:01.602754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:01.603844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 518/1040] 10.222.0.46 () {56 vars in 1120 bytes} [Thu May 26 18:05:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:01.609499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1ccd6-10a4-430e-b55c-5c52bedab59d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:01.720952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1ccd6-10a4-430e-b55c-5c52bedab59d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:01.728873 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:01.759275 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47e225-7423-492f-8aea-195b9d3097ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:01.759945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47e225-7423-492f-8aea-195b9d3097ff None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:01.760224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47e225-7423-492f-8aea-195b9d3097ff None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:01.760530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47e225-7423-492f-8aea-195b9d3097ff None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:01.761500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47e225-7423-492f-8aea-195b9d3097ff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:01.761843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47e225-7423-492f-8aea-195b9d3097ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:01.763243 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47e225-7423-492f-8aea-195b9d3097ff None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:01.776659 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 523/1041] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 18:05:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:01.777967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853fcddb-2025-4924-811f-78583a411dff None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:01.893505 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853fcddb-2025-4924-811f-78583a411dff None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:01.927654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877b4eb-ef22-49df-98e7-5c71917da419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:01.928448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877b4eb-ef22-49df-98e7-5c71917da419 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:01.928728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877b4eb-ef22-49df-98e7-5c71917da419 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:01.928994 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877b4eb-ef22-49df-98e7-5c71917da419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:01.940302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877b4eb-ef22-49df-98e7-5c71917da419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:01.941763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877b4eb-ef22-49df-98e7-5c71917da419 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f877b4eb-ef22-49df-98e7-5c71917da419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:05:01.958914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:01.959833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:01.960943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:05:01.960943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:05:01.960943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:01.960943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:01.960943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 519/1042] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:01.966016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47e225-7423-492f-8aea-195b9d3097ff None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:02.074280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47e225-7423-492f-8aea-195b9d3097ff None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:02.112016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ab195-0c2c-4d85-9bc8-c1e26485d3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:02.112700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ab195-0c2c-4d85-9bc8-c1e26485d3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:02.112981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ab195-0c2c-4d85-9bc8-c1e26485d3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:02.113253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ab195-0c2c-4d85-9bc8-c1e26485d3e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:02.114028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ab195-0c2c-4d85-9bc8-c1e26485d3e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:02.114334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ab195-0c2c-4d85-9bc8-c1e26485d3e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:02.115746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ab195-0c2c-4d85-9bc8-c1e26485d3e9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:02.129810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 524/1043] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:02.135343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877b4eb-ef22-49df-98e7-5c71917da419 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:02.171402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:02.249573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877b4eb-ef22-49df-98e7-5c71917da419 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:02.288736 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8951681-63b6-4447-9823-df7960f6a01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:02.289529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8951681-63b6-4447-9823-df7960f6a01a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:02.289890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8951681-63b6-4447-9823-df7960f6a01a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:02.290153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8951681-63b6-4447-9823-df7960f6a01a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:02.298245 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8951681-63b6-4447-9823-df7960f6a01a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:02.299686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8951681-63b6-4447-9823-df7960f6a01a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e8951681-63b6-4447-9823-df7960f6a01a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:05:02.317774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:02.318815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:02.319786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:02.319786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:02.319786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:05:02.319786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:05:02.319786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:05:02.319786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:05:02.319786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:05:02.319786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:05:02.319786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 18:05:02.319786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:02.321508 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ab195-0c2c-4d85-9bc8-c1e26485d3e9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:02.321980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 520/1044] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 18:05:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:02.364380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:02.441047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ab195-0c2c-4d85-9bc8-c1e26485d3e9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:02.477286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221618b-0770-4355-a379-75a0cd8a2e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:02.477989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221618b-0770-4355-a379-75a0cd8a2e77 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:02.478267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221618b-0770-4355-a379-75a0cd8a2e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:02.478534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221618b-0770-4355-a379-75a0cd8a2e77 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:02.479902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221618b-0770-4355-a379-75a0cd8a2e77 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:02.480212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221618b-0770-4355-a379-75a0cd8a2e77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:02.481660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221618b-0770-4355-a379-75a0cd8a2e77 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:02.504942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 525/1045] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 18:05:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:02.506454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8951681-63b6-4447-9823-df7960f6a01a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:02.616523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:02.626780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8951681-63b6-4447-9823-df7960f6a01a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:02.662077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d8dc2-a89c-4460-bc6c-157d32e1cf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:02.662727 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d8dc2-a89c-4460-bc6c-157d32e1cf5c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:02.663008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d8dc2-a89c-4460-bc6c-157d32e1cf5c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:02.663283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d8dc2-a89c-4460-bc6c-157d32e1cf5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:02.734947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d8dc2-a89c-4460-bc6c-157d32e1cf5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:02.736466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d8dc2-a89c-4460-bc6c-157d32e1cf5c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:02.831796 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:02.843967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 521/1046] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:05:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 18:05:02.849511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221618b-0770-4355-a379-75a0cd8a2e77 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:02.945865 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:02.966527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221618b-0770-4355-a379-75a0cd8a2e77 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:03.002147 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c69f69-e557-464f-a289-397c4a5466c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:03.002937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c69f69-e557-464f-a289-397c4a5466c1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.003214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c69f69-e557-464f-a289-397c4a5466c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.003480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c69f69-e557-464f-a289-397c4a5466c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.014134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c69f69-e557-464f-a289-397c4a5466c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:03.015612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c69f69-e557-464f-a289-397c4a5466c1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-e5c69f69-e557-464f-a289-397c4a5466c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:05:03.035299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:03.036289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:03.037445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:05:03.037445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:05:03.037445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:03.037445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:03.042352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 526/1047] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:03.044878 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d8dc2-a89c-4460-bc6c-157d32e1cf5c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:03.155415 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d8dc2-a89c-4460-bc6c-157d32e1cf5c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:03.192139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354e1b6-63b7-404f-9422-e06bf0a97061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:03.192804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354e1b6-63b7-404f-9422-e06bf0a97061 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.193084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354e1b6-63b7-404f-9422-e06bf0a97061 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.193357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354e1b6-63b7-404f-9422-e06bf0a97061 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.194140 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354e1b6-63b7-404f-9422-e06bf0a97061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:03.194554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354e1b6-63b7-404f-9422-e06bf0a97061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:03.196003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354e1b6-63b7-404f-9422-e06bf0a97061 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:03.213048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 522/1048] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:03.215374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c69f69-e557-464f-a289-397c4a5466c1 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:03.238857 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b0627-5cad-472b-bd3f-217aa678c531 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.239175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b0627-5cad-472b-bd3f-217aa678c531 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.239417 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b0627-5cad-472b-bd3f-217aa678c531 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.240946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 523/1049] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:03.246436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc3c8e-cfd0-4aa4-8e61-ef88b66866f8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.246717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc3c8e-cfd0-4aa4-8e61-ef88b66866f8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.246989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc3c8e-cfd0-4aa4-8e61-ef88b66866f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.263864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-75dc3c8e-cfd0-4aa4-8e61-ef88b66866f8 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:03.265062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-75dc3c8e-cfd0-4aa4-8e61-ef88b66866f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:03.324848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:03.333049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c69f69-e557-464f-a289-397c4a5466c1 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:03.372453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 524/1050] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:05:03.382480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518e73e-15ae-4509-8f9d-31fc8209dd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:03.383269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518e73e-15ae-4509-8f9d-31fc8209dd19 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.383546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518e73e-15ae-4509-8f9d-31fc8209dd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.383813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518e73e-15ae-4509-8f9d-31fc8209dd19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.390803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518e73e-15ae-4509-8f9d-31fc8209dd19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:03.392273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518e73e-15ae-4509-8f9d-31fc8209dd19 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:03.394222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:03.395828 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc3c8e-cfd0-4aa4-8e61-ef88b66866f8 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-0518e73e-15ae-4509-8f9d-31fc8209dd19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:05:03.408706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:03.409878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:03.410798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:05:03.410798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:05:03.410798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:05:03.410798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:05:03.410798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:05:03.410798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:05:03.410798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:05:03.410798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:03.419851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 527/1051] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:05:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:03.421461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518e73e-15ae-4509-8f9d-31fc8209dd19 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:03.511855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc3c8e-cfd0-4aa4-8e61-ef88b66866f8 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:03.537068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518e73e-15ae-4509-8f9d-31fc8209dd19 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:03.563341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f7721-f071-4d08-8648-66806055ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:03.564161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f7721-f071-4d08-8648-66806055ff1e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.564444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f7721-f071-4d08-8648-66806055ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.564715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f7721-f071-4d08-8648-66806055ff1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.576457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f7721-f071-4d08-8648-66806055ff1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-6b4f7721-f071-4d08-8648-66806055ff1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:05:03.584050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:03.585128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:03.586701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 525/1052] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:05:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:03.587633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347a0a9-e8ae-4026-96f5-f8d9e0ae6d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:03.588375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347a0a9-e8ae-4026-96f5-f8d9e0ae6d96 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.589299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347a0a9-e8ae-4026-96f5-f8d9e0ae6d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.589561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347a0a9-e8ae-4026-96f5-f8d9e0ae6d96 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.590389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347a0a9-e8ae-4026-96f5-f8d9e0ae6d96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:03.591025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347a0a9-e8ae-4026-96f5-f8d9e0ae6d96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:03.591721 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f7721-f071-4d08-8648-66806055ff1e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:03.592493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347a0a9-e8ae-4026-96f5-f8d9e0ae6d96 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:03.611232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 528/1053] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:05:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:03.612816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347a0a9-e8ae-4026-96f5-f8d9e0ae6d96 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:03.640504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:03.671685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:03.697077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f7721-f071-4d08-8648-66806055ff1e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:03.712884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347a0a9-e8ae-4026-96f5-f8d9e0ae6d96 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:03.730740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e26c4-ba5e-4f28-a169-c2fec5365e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:03.731408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e26c4-ba5e-4f28-a169-c2fec5365e7f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.731689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e26c4-ba5e-4f28-a169-c2fec5365e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.732040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e26c4-ba5e-4f28-a169-c2fec5365e7f None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.732707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e26c4-ba5e-4f28-a169-c2fec5365e7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:03.733012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e26c4-ba5e-4f28-a169-c2fec5365e7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:03.734421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e26c4-ba5e-4f28-a169-c2fec5365e7f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:03.745215 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 526/1054] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:05:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:03.747871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985d4c2-6b6a-40b3-bc29-2019148d487d None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.748169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985d4c2-6b6a-40b3-bc29-2019148d487d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.748336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e657d-147a-4223-8a55-b3418db5261b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:03.748454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985d4c2-6b6a-40b3-bc29-2019148d487d None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.748952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e657d-147a-4223-8a55-b3418db5261b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.749232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e657d-147a-4223-8a55-b3418db5261b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.749503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e657d-147a-4223-8a55-b3418db5261b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.753621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 527/1055] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:03.753793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e657d-147a-4223-8a55-b3418db5261b None admin] RBAC: Adding query filter params (role.id=e9715685617b47c7809732b0a8c61c35, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=d420e808730d484f80963322c5fc6856) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:03.753793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e657d-147a-4223-8a55-b3418db5261b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:03.753963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985d4c2-6b6a-40b3-bc29-2019148d487d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:03.754034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e657d-147a-4223-8a55-b3418db5261b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:03.766171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 529/1056] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:05:03 2022] GET /identity/v3/role_assignments?user.id=d420e808730d484f80963322c5fc6856&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=e9715685617b47c7809732b0a8c61c35 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:03.766171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d746e-6942-4164-84f8-2e16356c9a93 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.766171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d746e-6942-4164-84f8-2e16356c9a93 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.766171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d746e-6942-4164-84f8-2e16356c9a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.778228 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-348d746e-6942-4164-84f8-2e16356c9a93 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:03.779529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-348d746e-6942-4164-84f8-2e16356c9a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:03.865040 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 530/1057] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:03.887444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d746e-6942-4164-84f8-2e16356c9a93 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:03.889204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985d4c2-6b6a-40b3-bc29-2019148d487d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:03.892125 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:03.931687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754647b8-0e48-4ae5-a59c-f372ce1e2f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:03.933189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754647b8-0e48-4ae5-a59c-f372ce1e2f3c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.933635 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754647b8-0e48-4ae5-a59c-f372ce1e2f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.934189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754647b8-0e48-4ae5-a59c-f372ce1e2f3c None admin] PATH_INFO: `/v3/users/d0fcab5e78564b4e8463d4194e70dd83` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.953420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754647b8-0e48-4ae5-a59c-f372ce1e2f3c None admin] RBAC: Authorizing `identity:get_user(user_id=d0fcab5e78564b4e8463d4194e70dd83)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:03.954585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:03.954906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754647b8-0e48-4ae5-a59c-f372ce1e2f3c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:03.956926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 528/1058] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:03 2022] GET /identity/v3/users/d0fcab5e78564b4e8463d4194e70dd83 => generated 285 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:03.962329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754647b8-0e48-4ae5-a59c-f372ce1e2f3c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:04.002865 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d746e-6942-4164-84f8-2e16356c9a93 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:04.038224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9812fb0-d119-4430-95b5-92bda66e7b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:04.038792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9812fb0-d119-4430-95b5-92bda66e7b3a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:04.039591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9812fb0-d119-4430-95b5-92bda66e7b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:04.039874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9812fb0-d119-4430-95b5-92bda66e7b3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:04.071605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754647b8-0e48-4ae5-a59c-f372ce1e2f3c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:04.113000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c69ae-b8af-47cc-9a9a-b2fa202847e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:04.113652 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c69ae-b8af-47cc-9a9a-b2fa202847e2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:04.114050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c69ae-b8af-47cc-9a9a-b2fa202847e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:04.114337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c69ae-b8af-47cc-9a9a-b2fa202847e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:04.118250 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9812fb0-d119-4430-95b5-92bda66e7b3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:04.119725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9812fb0-d119-4430-95b5-92bda66e7b3a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:04.149022 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:04.178730 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:04.190907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c69ae-b8af-47cc-9a9a-b2fa202847e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:04.192397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c69ae-b8af-47cc-9a9a-b2fa202847e2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:04.231617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 531/1059] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:05:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 18:05:04.239122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9812fb0-d119-4430-95b5-92bda66e7b3a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:04.313827 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 529/1060] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:05:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:05:04.319611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c69ae-b8af-47cc-9a9a-b2fa202847e2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:04.355592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9812fb0-d119-4430-95b5-92bda66e7b3a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:04.386527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:04.392630 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50d6eb-9798-4aa4-aceb-a15578f234ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:04.393419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50d6eb-9798-4aa4-aceb-a15578f234ec None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:04.393690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50d6eb-9798-4aa4-aceb-a15578f234ec None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:04.393987 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50d6eb-9798-4aa4-aceb-a15578f234ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:04.404849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50d6eb-9798-4aa4-aceb-a15578f234ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:04.406359 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50d6eb-9798-4aa4-aceb-a15578f234ec None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-3d50d6eb-9798-4aa4-aceb-a15578f234ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:05:04.426616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:04.427559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:04.428673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:05:04.428673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:05:04.428673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:04.428673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:04.428673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 532/1061] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:04.433640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50d6eb-9798-4aa4-aceb-a15578f234ec None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:04.435380 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c69ae-b8af-47cc-9a9a-b2fa202847e2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:04.437243 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:04.477752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46c614-10ae-4da0-a552-77f375a7db9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:04.478965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46c614-10ae-4da0-a552-77f375a7db9c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:04.479469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46c614-10ae-4da0-a552-77f375a7db9c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:04.480130 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46c614-10ae-4da0-a552-77f375a7db9c None admin] PATH_INFO: `/v3/projects/3547af6a2ae6426baefee43df7476d2b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:04.481496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46c614-10ae-4da0-a552-77f375a7db9c None admin] RBAC: Authorizing `identity:get_project(project_id=3547af6a2ae6426baefee43df7476d2b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:04.483099 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46c614-10ae-4da0-a552-77f375a7db9c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:04.484956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 530/1062] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:05:04 2022] GET /identity/v3/projects/3547af6a2ae6426baefee43df7476d2b => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:04.490871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46c614-10ae-4da0-a552-77f375a7db9c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:04.546818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50d6eb-9798-4aa4-aceb-a15578f234ec None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:04.585412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02c55d-91e4-4502-9b17-10f0df14ef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:04.586019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02c55d-91e4-4502-9b17-10f0df14ef0b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:04.586414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02c55d-91e4-4502-9b17-10f0df14ef0b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:04.586687 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02c55d-91e4-4502-9b17-10f0df14ef0b None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:04.587523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02c55d-91e4-4502-9b17-10f0df14ef0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:04.587827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02c55d-91e4-4502-9b17-10f0df14ef0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:04.589984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02c55d-91e4-4502-9b17-10f0df14ef0b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:04.596409 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46c614-10ae-4da0-a552-77f375a7db9c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:04.609835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 533/1063] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:04.635330 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:04.642858 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37f660-6ea6-48e6-999a-eaa22fe75857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:04.643645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37f660-6ea6-48e6-999a-eaa22fe75857 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:04.644018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37f660-6ea6-48e6-999a-eaa22fe75857 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:04.644351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37f660-6ea6-48e6-999a-eaa22fe75857 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:04.645252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37f660-6ea6-48e6-999a-eaa22fe75857 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=3547af6a2ae6426baefee43df7476d2b, user.id=d0fcab5e78564b4e8463d4194e70dd83) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:04.645627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37f660-6ea6-48e6-999a-eaa22fe75857 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:04.647196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37f660-6ea6-48e6-999a-eaa22fe75857 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:04.654867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 531/1064] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:05:04 2022] GET /identity/v3/role_assignments?user.id=d0fcab5e78564b4e8463d4194e70dd83&scope.project.id=3547af6a2ae6426baefee43df7476d2b&role.id=584beecb022b422f820e77f7623ada5e => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:05.996437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2884c-3444-478e-9d4f-2a40345c0452 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:05.996964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2884c-3444-478e-9d4f-2a40345c0452 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:05.996964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2884c-3444-478e-9d4f-2a40345c0452 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:05.998358 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 534/1065] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:06.003613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb9a7e-d064-4250-96d9-978245a097c0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:06.004044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb9a7e-d064-4250-96d9-978245a097c0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:06.004295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb9a7e-d064-4250-96d9-978245a097c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:06.021251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e0fb9a7e-d064-4250-96d9-978245a097c0 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:06.022469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e0fb9a7e-d064-4250-96d9-978245a097c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:06.096469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 532/1066] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:06.122101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2884c-3444-478e-9d4f-2a40345c0452 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:06.186506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:06.231625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2884c-3444-478e-9d4f-2a40345c0452 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:06.270614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c217f4-0a6d-4289-9784-320e2e6d9ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:06.271352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c217f4-0a6d-4289-9784-320e2e6d9ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:06.271630 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c217f4-0a6d-4289-9784-320e2e6d9ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:06.271897 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c217f4-0a6d-4289-9784-320e2e6d9ae0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:06.286781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c217f4-0a6d-4289-9784-320e2e6d9ae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-98c217f4-0a6d-4289-9784-320e2e6d9ae0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:05:06.297117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:06.298249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:06.299114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 535/1067] 10.222.0.46 () {56 vars in 1120 bytes} [Thu May 26 18:05:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:06.305301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb9a7e-d064-4250-96d9-978245a097c0 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:06.410343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb9a7e-d064-4250-96d9-978245a097c0 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:06.435480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:06.446259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119983e-21b2-4752-9c98-638d71f7e854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:06.446926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119983e-21b2-4752-9c98-638d71f7e854 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:06.447205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119983e-21b2-4752-9c98-638d71f7e854 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:06.447475 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119983e-21b2-4752-9c98-638d71f7e854 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:06.448453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119983e-21b2-4752-9c98-638d71f7e854 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:06.448848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119983e-21b2-4752-9c98-638d71f7e854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:06.450411 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119983e-21b2-4752-9c98-638d71f7e854 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:06.458706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 533/1068] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 18:05:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:06.465273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c217f4-0a6d-4289-9784-320e2e6d9ae0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:06.567960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c217f4-0a6d-4289-9784-320e2e6d9ae0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:06.591859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:06.603824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2922e2-9bf1-4f8a-a8ac-d96e5b52637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:06.605276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2922e2-9bf1-4f8a-a8ac-d96e5b52637a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:06.605552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2922e2-9bf1-4f8a-a8ac-d96e5b52637a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:06.605854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2922e2-9bf1-4f8a-a8ac-d96e5b52637a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:06.616832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2922e2-9bf1-4f8a-a8ac-d96e5b52637a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:06.618333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2922e2-9bf1-4f8a-a8ac-d96e5b52637a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-ed2922e2-9bf1-4f8a-a8ac-d96e5b52637a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:05:06.636770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:06.637973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:06.640240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:05:06.640240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:05:06.640240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:06.640240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:06.641957 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 536/1069] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:06.644185 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119983e-21b2-4752-9c98-638d71f7e854 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:06.747761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119983e-21b2-4752-9c98-638d71f7e854 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:06.783115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651cccd-3df5-4530-8b6a-f62b1b092510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:06.783775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651cccd-3df5-4530-8b6a-f62b1b092510 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:06.784054 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651cccd-3df5-4530-8b6a-f62b1b092510 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:06.784322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651cccd-3df5-4530-8b6a-f62b1b092510 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:06.785030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651cccd-3df5-4530-8b6a-f62b1b092510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:06.785334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651cccd-3df5-4530-8b6a-f62b1b092510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:06.786984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651cccd-3df5-4530-8b6a-f62b1b092510 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:06.799709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 534/1070] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:06.805442 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2922e2-9bf1-4f8a-a8ac-d96e5b52637a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:06.914023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2922e2-9bf1-4f8a-a8ac-d96e5b52637a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:06.931034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c80461-8ad1-4205-8f3c-853f65ad733f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:06.931320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c80461-8ad1-4205-8f3c-853f65ad733f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:06.931592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c80461-8ad1-4205-8f3c-853f65ad733f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:06.933485 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 535/1071] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:05:06.938728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0fec5-388b-4773-aacb-47d7baf7a303 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:06.939009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0fec5-388b-4773-aacb-47d7baf7a303 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:06.939279 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0fec5-388b-4773-aacb-47d7baf7a303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:06.953172 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba0d9e-c332-492b-96ae-6391546b8c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:06.953985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba0d9e-c332-492b-96ae-6391546b8c43 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:06.954269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba0d9e-c332-492b-96ae-6391546b8c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:06.954541 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba0d9e-c332-492b-96ae-6391546b8c43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:06.955488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-3fd0fec5-388b-4773-aacb-47d7baf7a303 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:06.956666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3fd0fec5-388b-4773-aacb-47d7baf7a303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:06.962286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba0d9e-c332-492b-96ae-6391546b8c43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:06.964382 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba0d9e-c332-492b-96ae-6391546b8c43 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-75ba0d9e-c332-492b-96ae-6391546b8c43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:05:06.978611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:06.980194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:06.981245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:06.981245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:06.981245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:05:06.981245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:05:06.981245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:05:06.981245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:05:06.981245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:05:06.981245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:05:06.981245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 18:05:06.981245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:06.981245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 537/1072] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 18:05:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:06.986112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba0d9e-c332-492b-96ae-6391546b8c43 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:07.001406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:07.022915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:07.028740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 536/1073] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:05:07.052332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0fec5-388b-4773-aacb-47d7baf7a303 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:07.116789 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba0d9e-c332-492b-96ae-6391546b8c43 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:07.154156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2236a13-1f04-4303-9fe6-449f5715feb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:07.154820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2236a13-1f04-4303-9fe6-449f5715feb5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:07.155101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2236a13-1f04-4303-9fe6-449f5715feb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:07.155372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2236a13-1f04-4303-9fe6-449f5715feb5 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:07.156199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2236a13-1f04-4303-9fe6-449f5715feb5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:07.156508 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2236a13-1f04-4303-9fe6-449f5715feb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:07.157984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2236a13-1f04-4303-9fe6-449f5715feb5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:07.177277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 538/1074] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 18:05:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:07.178421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0fec5-388b-4773-aacb-47d7baf7a303 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:07.183974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2236a13-1f04-4303-9fe6-449f5715feb5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:07.217339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7ab01-021c-47f6-8f6b-9846cbf51dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:07.218169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7ab01-021c-47f6-8f6b-9846cbf51dea None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:07.218463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7ab01-021c-47f6-8f6b-9846cbf51dea None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:07.218724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7ab01-021c-47f6-8f6b-9846cbf51dea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:07.228974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7ab01-021c-47f6-8f6b-9846cbf51dea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-28d7ab01-021c-47f6-8f6b-9846cbf51dea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:05:07.237744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:07.238920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:07.239780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 537/1075] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:07.250616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7ab01-021c-47f6-8f6b-9846cbf51dea None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:07.287444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:07.298918 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:07.306724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2236a13-1f04-4303-9fe6-449f5715feb5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:07.344461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4450f6-5d91-4d36-ae5e-c16eba9f04c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:07.345240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4450f6-5d91-4d36-ae5e-c16eba9f04c6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:07.345511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4450f6-5d91-4d36-ae5e-c16eba9f04c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:07.345796 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4450f6-5d91-4d36-ae5e-c16eba9f04c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:07.356350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4450f6-5d91-4d36-ae5e-c16eba9f04c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:07.357868 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4450f6-5d91-4d36-ae5e-c16eba9f04c6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:07.371768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7ab01-021c-47f6-8f6b-9846cbf51dea None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-7b4450f6-5d91-4d36-ae5e-c16eba9f04c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:05:07.376002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:07.377087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:07.378360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:05:07.378360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:05:07.378360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:07.378360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:07.378360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 539/1076] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:07.382818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4450f6-5d91-4d36-ae5e-c16eba9f04c6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:07.408324 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f67aae-d3ca-4229-8778-788468449a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:07.408984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f67aae-d3ca-4229-8778-788468449a62 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:07.409264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f67aae-d3ca-4229-8778-788468449a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:07.409534 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f67aae-d3ca-4229-8778-788468449a62 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:07.410301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f67aae-d3ca-4229-8778-788468449a62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:07.410609 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f67aae-d3ca-4229-8778-788468449a62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:07.412243 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f67aae-d3ca-4229-8778-788468449a62 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:07.421489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 538/1077] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:05:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:07.427409 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f67aae-d3ca-4229-8778-788468449a62 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:07.497505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4450f6-5d91-4d36-ae5e-c16eba9f04c6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:07.534333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0306f55-b83a-4d60-b4f7-f739c2679911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:07.534594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:07.535000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0306f55-b83a-4d60-b4f7-f739c2679911 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:07.535276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0306f55-b83a-4d60-b4f7-f739c2679911 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:07.535619 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0306f55-b83a-4d60-b4f7-f739c2679911 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:07.536392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0306f55-b83a-4d60-b4f7-f739c2679911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:07.536991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0306f55-b83a-4d60-b4f7-f739c2679911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:07.538450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0306f55-b83a-4d60-b4f7-f739c2679911 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:07.538962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f67aae-d3ca-4229-8778-788468449a62 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:07.550784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 540/1078] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:07.556759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0306f55-b83a-4d60-b4f7-f739c2679911 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:07.566447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:07.579095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed272b48-01e4-4be6-934f-24010fdc7f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:07.579968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed272b48-01e4-4be6-934f-24010fdc7f22 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:07.580243 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed272b48-01e4-4be6-934f-24010fdc7f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:07.580512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed272b48-01e4-4be6-934f-24010fdc7f22 None admin] PATH_INFO: `/v3/users/d0fcab5e78564b4e8463d4194e70dd83` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:07.581701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed272b48-01e4-4be6-934f-24010fdc7f22 None admin] RBAC: Authorizing `identity:get_user(user_id=d0fcab5e78564b4e8463d4194e70dd83)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:07.583154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed272b48-01e4-4be6-934f-24010fdc7f22 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:07.585098 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 539/1079] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:07 2022] GET /identity/v3/users/d0fcab5e78564b4e8463d4194e70dd83 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:07.590787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed272b48-01e4-4be6-934f-24010fdc7f22 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:07.659705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0306f55-b83a-4d60-b4f7-f739c2679911 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:07.691206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed272b48-01e4-4be6-934f-24010fdc7f22 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:07.693722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d7d0d-c69d-44b8-8389-d73e46ab44ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:07.694525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d7d0d-c69d-44b8-8389-d73e46ab44ff None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:07.694799 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d7d0d-c69d-44b8-8389-d73e46ab44ff None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:07.695066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d7d0d-c69d-44b8-8389-d73e46ab44ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:07.702223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d7d0d-c69d-44b8-8389-d73e46ab44ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:07.704082 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d7d0d-c69d-44b8-8389-d73e46ab44ff None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-6b9d7d0d-c69d-44b8-8389-d73e46ab44ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:05:07.719289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:07.720294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:07.721210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:05:07.721210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:05:07.721210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:05:07.721210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:05:07.721210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:05:07.721210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:05:07.721210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:05:07.721210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:07.721210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 541/1080] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:05:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:07.726420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d7d0d-c69d-44b8-8389-d73e46ab44ff None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:07.727893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734217c-4b08-413b-92f6-ba4e5d31f55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:07.728774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734217c-4b08-413b-92f6-ba4e5d31f55b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:07.729084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734217c-4b08-413b-92f6-ba4e5d31f55b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:07.730005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734217c-4b08-413b-92f6-ba4e5d31f55b None admin] PATH_INFO: `/v3/projects/3547af6a2ae6426baefee43df7476d2b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:07.731276 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734217c-4b08-413b-92f6-ba4e5d31f55b None admin] RBAC: Authorizing `identity:get_project(project_id=3547af6a2ae6426baefee43df7476d2b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:07.732757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734217c-4b08-413b-92f6-ba4e5d31f55b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:07.734599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 540/1081] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:05:07 2022] GET /identity/v3/projects/3547af6a2ae6426baefee43df7476d2b => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:07.740636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734217c-4b08-413b-92f6-ba4e5d31f55b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:07.775696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:07.795954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:07.829837 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d7d0d-c69d-44b8-8389-d73e46ab44ff None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:07.845409 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734217c-4b08-413b-92f6-ba4e5d31f55b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:07.865108 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d919ed-4f88-4ce7-9ae3-20d4efb00ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:07.865885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d919ed-4f88-4ce7-9ae3-20d4efb00ddc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:07.866227 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d919ed-4f88-4ce7-9ae3-20d4efb00ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:07.866504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d919ed-4f88-4ce7-9ae3-20d4efb00ddc None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:07.867283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d919ed-4f88-4ce7-9ae3-20d4efb00ddc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:07.867588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d919ed-4f88-4ce7-9ae3-20d4efb00ddc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:07.869025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d919ed-4f88-4ce7-9ae3-20d4efb00ddc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:07.879313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdabec-0222-4571-a357-12227b96a384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:07.880188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdabec-0222-4571-a357-12227b96a384 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:07.880684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdabec-0222-4571-a357-12227b96a384 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:07.880805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 542/1082] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:05:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:07.880895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdabec-0222-4571-a357-12227b96a384 None admin] PATH_INFO: `/v3/projects/3547af6a2ae6426baefee43df7476d2b/users/d0fcab5e78564b4e8463d4194e70dd83/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:07.882701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdabec-0222-4571-a357-12227b96a384 None admin] RBAC: Authorizing `identity:create_grant(project_id=3547af6a2ae6426baefee43df7476d2b, user_id=d0fcab5e78564b4e8463d4194e70dd83, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:07.884116 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdabec-0222-4571-a357-12227b96a384 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:07.886389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d919ed-4f88-4ce7-9ae3-20d4efb00ddc None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:07.897580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 541/1083] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:05:07 2022] PUT /identity/v3/projects/3547af6a2ae6426baefee43df7476d2b/users/d0fcab5e78564b4e8463d4194e70dd83/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:05:07.996360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d919ed-4f88-4ce7-9ae3-20d4efb00ddc None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:08.022994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:08.034080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a683438-4bcb-4dff-8e0a-34a8a7952d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:08.034991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a683438-4bcb-4dff-8e0a-34a8a7952d5e None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:08.035266 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a683438-4bcb-4dff-8e0a-34a8a7952d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:08.035536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a683438-4bcb-4dff-8e0a-34a8a7952d5e None admin] PATH_INFO: `/v3/projects/4be4045293754773bb52bbb82546a1db/users/d420e808730d484f80963322c5fc6856/roles/e9715685617b47c7809732b0a8c61c35` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:08.038156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a683438-4bcb-4dff-8e0a-34a8a7952d5e None admin] RBAC: Authorizing `identity:create_grant(project_id=4be4045293754773bb52bbb82546a1db, user_id=d420e808730d484f80963322c5fc6856, role_id=e9715685617b47c7809732b0a8c61c35)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:08.039642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a683438-4bcb-4dff-8e0a-34a8a7952d5e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:08.055139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 543/1084] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:05:07 2022] PUT /identity/v3/projects/4be4045293754773bb52bbb82546a1db/users/d420e808730d484f80963322c5fc6856/roles/e9715685617b47c7809732b0a8c61c35 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:05:10.152943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68549a-c6fd-400f-803e-ce84a6cbe988 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:10.153507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68549a-c6fd-400f-803e-ce84a6cbe988 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:10.153507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68549a-c6fd-400f-803e-ce84a6cbe988 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:10.154899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 542/1085] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:10.160321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2b798-1190-402a-a5ae-d25ca44a3a21 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:10.160725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2b798-1190-402a-a5ae-d25ca44a3a21 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:10.160863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2b798-1190-402a-a5ae-d25ca44a3a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:10.177780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-77a2b798-1190-402a-a5ae-d25ca44a3a21 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:10.179079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-77a2b798-1190-402a-a5ae-d25ca44a3a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:10.246574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 544/1086] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:10.270330 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab68549a-c6fd-400f-803e-ce84a6cbe988 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:10.306713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb7981-ca0d-4f9f-9fe6-14e665e76951 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:10.306997 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb7981-ca0d-4f9f-9fe6-14e665e76951 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:10.307322 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb7981-ca0d-4f9f-9fe6-14e665e76951 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:10.309087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 545/1087] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:05:10.313999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425135b-5d92-486b-ae0a-d53386e5f0b0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:10.317038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425135b-5d92-486b-ae0a-d53386e5f0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:10.317038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425135b-5d92-486b-ae0a-d53386e5f0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:10.333339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8425135b-5d92-486b-ae0a-d53386e5f0b0 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:10.334570 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8425135b-5d92-486b-ae0a-d53386e5f0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:10.368806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:10.385304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab68549a-c6fd-400f-803e-ce84a6cbe988 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:10.403233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 546/1088] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:05:10.424794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020ef6f-990f-4e69-98c9-ee52cdb67acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:10.425586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020ef6f-990f-4e69-98c9-ee52cdb67acd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:10.425905 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020ef6f-990f-4e69-98c9-ee52cdb67acd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:10.426185 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020ef6f-990f-4e69-98c9-ee52cdb67acd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:10.427380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425135b-5d92-486b-ae0a-d53386e5f0b0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:10.437027 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020ef6f-990f-4e69-98c9-ee52cdb67acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-6020ef6f-990f-4e69-98c9-ee52cdb67acd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:05:10.444869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:10.446472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:10.447322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 543/1089] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:05:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:10.453088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020ef6f-990f-4e69-98c9-ee52cdb67acd None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:10.478755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:10.537648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425135b-5d92-486b-ae0a-d53386e5f0b0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:10.556830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020ef6f-990f-4e69-98c9-ee52cdb67acd None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:10.571306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fc86e-589b-4061-9e42-a3a03b198752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:10.572104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fc86e-589b-4061-9e42-a3a03b198752 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:10.572378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fc86e-589b-4061-9e42-a3a03b198752 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:10.572645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fc86e-589b-4061-9e42-a3a03b198752 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:10.583838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fc86e-589b-4061-9e42-a3a03b198752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-913fc86e-589b-4061-9e42-a3a03b198752 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:05:10.592649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:10.593883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:10.594907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 547/1090] 10.222.0.46 () {56 vars in 1120 bytes} [Thu May 26 18:05:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:10.599939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fc86e-589b-4061-9e42-a3a03b198752 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:10.600082 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a999eca-3329-475c-843d-c358b6c4a991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:10.600614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a999eca-3329-475c-843d-c358b6c4a991 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:10.600903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a999eca-3329-475c-843d-c358b6c4a991 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:10.601176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a999eca-3329-475c-843d-c358b6c4a991 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:10.601956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a999eca-3329-475c-843d-c358b6c4a991 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:10.602266 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a999eca-3329-475c-843d-c358b6c4a991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:10.603642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a999eca-3329-475c-843d-c358b6c4a991 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:10.613270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 544/1091] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:05:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:10.618967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a999eca-3329-475c-843d-c358b6c4a991 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:10.638161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:10.706214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fc86e-589b-4061-9e42-a3a03b198752 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:10.713085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:10.722193 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a999eca-3329-475c-843d-c358b6c4a991 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:10.740979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5afca7-2a80-4cd9-890a-897e199b4c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:10.741643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5afca7-2a80-4cd9-890a-897e199b4c66 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:10.741952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5afca7-2a80-4cd9-890a-897e199b4c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:10.742223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5afca7-2a80-4cd9-890a-897e199b4c66 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:10.742967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5afca7-2a80-4cd9-890a-897e199b4c66 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:10.743273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5afca7-2a80-4cd9-890a-897e199b4c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:10.745174 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5afca7-2a80-4cd9-890a-897e199b4c66 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:10.753043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 548/1092] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 18:05:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:10.755919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a00c69-8358-4834-b354-a2da5c001aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:10.756785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a00c69-8358-4834-b354-a2da5c001aaa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:10.758768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a00c69-8358-4834-b354-a2da5c001aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:10.758768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a00c69-8358-4834-b354-a2da5c001aaa None admin] PATH_INFO: `/v3/users/d0fcab5e78564b4e8463d4194e70dd83` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:10.759133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5afca7-2a80-4cd9-890a-897e199b4c66 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:10.759445 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a00c69-8358-4834-b354-a2da5c001aaa None admin] RBAC: Authorizing `identity:get_user(user_id=d0fcab5e78564b4e8463d4194e70dd83)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:10.761437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a00c69-8358-4834-b354-a2da5c001aaa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:10.763881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 545/1093] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:10 2022] GET /identity/v3/users/d0fcab5e78564b4e8463d4194e70dd83 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:10.770004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a00c69-8358-4834-b354-a2da5c001aaa None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:10.860947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:10.877385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5afca7-2a80-4cd9-890a-897e199b4c66 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:10.877709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a00c69-8358-4834-b354-a2da5c001aaa None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:10.910699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e9427-b4d8-43ab-a99d-64af76b911b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:10.911498 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e9427-b4d8-43ab-a99d-64af76b911b7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:10.912033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e9427-b4d8-43ab-a99d-64af76b911b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:10.912156 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431ca5a-b5f6-47c2-b679-e30d67cadaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:10.912325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e9427-b4d8-43ab-a99d-64af76b911b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:10.912800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431ca5a-b5f6-47c2-b679-e30d67cadaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:10.913078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431ca5a-b5f6-47c2-b679-e30d67cadaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:10.913351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431ca5a-b5f6-47c2-b679-e30d67cadaf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:10.923262 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e9427-b4d8-43ab-a99d-64af76b911b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:10.924745 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e9427-b4d8-43ab-a99d-64af76b911b7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-527e9427-b4d8-43ab-a99d-64af76b911b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:05:10.942522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:10.943607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:10.949995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:05:10.949995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:05:10.949995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:10.949995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:10.949995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 549/1094] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:10.950287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e9427-b4d8-43ab-a99d-64af76b911b7 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:10.972595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:10.988259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431ca5a-b5f6-47c2-b679-e30d67cadaf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:10.989825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431ca5a-b5f6-47c2-b679-e30d67cadaf2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:11.065751 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e9427-b4d8-43ab-a99d-64af76b911b7 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:11.087196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:11.101620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88c853-9fc1-433d-8f9d-199560e28670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:11.102557 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88c853-9fc1-433d-8f9d-199560e28670 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:11.102841 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88c853-9fc1-433d-8f9d-199560e28670 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:11.103118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88c853-9fc1-433d-8f9d-199560e28670 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:11.103399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 546/1095] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:05:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 18:05:11.103839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88c853-9fc1-433d-8f9d-199560e28670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:11.104779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88c853-9fc1-433d-8f9d-199560e28670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:11.106301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88c853-9fc1-433d-8f9d-199560e28670 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:11.109660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431ca5a-b5f6-47c2-b679-e30d67cadaf2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:11.118716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 550/1096] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:11.124768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88c853-9fc1-433d-8f9d-199560e28670 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:11.213175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:11.214146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431ca5a-b5f6-47c2-b679-e30d67cadaf2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:11.233956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88c853-9fc1-433d-8f9d-199560e28670 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:11.248695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3105917-13a5-43ee-867c-8377113f8e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:11.249565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3105917-13a5-43ee-867c-8377113f8e92 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:11.249855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3105917-13a5-43ee-867c-8377113f8e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:11.250128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3105917-13a5-43ee-867c-8377113f8e92 None admin] PATH_INFO: `/v3/projects/3547af6a2ae6426baefee43df7476d2b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:11.251298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3105917-13a5-43ee-867c-8377113f8e92 None admin] RBAC: Authorizing `identity:get_project(project_id=3547af6a2ae6426baefee43df7476d2b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:11.252714 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3105917-13a5-43ee-867c-8377113f8e92 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:11.254440 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 547/1097] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:05:11 2022] GET /identity/v3/projects/3547af6a2ae6426baefee43df7476d2b => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:11.260382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3105917-13a5-43ee-867c-8377113f8e92 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:11.267606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1f340-c744-442b-9c92-0fa7e62b172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:11.268397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1f340-c744-442b-9c92-0fa7e62b172f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:11.268685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1f340-c744-442b-9c92-0fa7e62b172f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:11.268958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1f340-c744-442b-9c92-0fa7e62b172f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:11.277381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1f340-c744-442b-9c92-0fa7e62b172f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:11.278872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1f340-c744-442b-9c92-0fa7e62b172f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-04e1f340-c744-442b-9c92-0fa7e62b172f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:05:11.294022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:11.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:11.295910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:11.295910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:11.295910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:05:11.295910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:05:11.295910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:05:11.295910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:05:11.295910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:05:11.295910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:05:11.295910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 18:05:11.295910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:11.295910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 551/1098] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 18:05:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:11.300734 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1f340-c744-442b-9c92-0fa7e62b172f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:11.320975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:11.367709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3105917-13a5-43ee-867c-8377113f8e92 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:11.400125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1f340-c744-442b-9c92-0fa7e62b172f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:11.402828 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d322b-6739-4952-8eda-10c0ddf11aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:11.403477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d322b-6739-4952-8eda-10c0ddf11aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:11.403759 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d322b-6739-4952-8eda-10c0ddf11aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:11.404034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d322b-6739-4952-8eda-10c0ddf11aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:11.404848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d322b-6739-4952-8eda-10c0ddf11aa1 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=3547af6a2ae6426baefee43df7476d2b, user.id=d0fcab5e78564b4e8463d4194e70dd83) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:11.405159 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d322b-6739-4952-8eda-10c0ddf11aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:11.406649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d322b-6739-4952-8eda-10c0ddf11aa1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:11.413072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 548/1099] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:05:11 2022] GET /identity/v3/role_assignments?user.id=d0fcab5e78564b4e8463d4194e70dd83&scope.project.id=3547af6a2ae6426baefee43df7476d2b&role.id=584beecb022b422f820e77f7623ada5e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:11.437171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b83493-d845-4a23-9cf0-fa0b34472e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:11.437895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b83493-d845-4a23-9cf0-fa0b34472e1a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:11.438174 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b83493-d845-4a23-9cf0-fa0b34472e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:11.438444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b83493-d845-4a23-9cf0-fa0b34472e1a None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:11.439292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b83493-d845-4a23-9cf0-fa0b34472e1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:11.439599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b83493-d845-4a23-9cf0-fa0b34472e1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:11.441088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b83493-d845-4a23-9cf0-fa0b34472e1a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:11.452047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:11.462277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 552/1100] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 18:05:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:11.469205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d322b-6739-4952-8eda-10c0ddf11aa1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:11.577906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d322b-6739-4952-8eda-10c0ddf11aa1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:11.594890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:11.616826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fd7c4-b0b7-45cd-9cc2-180f5ca46161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:11.617477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fd7c4-b0b7-45cd-9cc2-180f5ca46161 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:11.617780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fd7c4-b0b7-45cd-9cc2-180f5ca46161 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:11.618054 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fd7c4-b0b7-45cd-9cc2-180f5ca46161 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:11.685692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fd7c4-b0b7-45cd-9cc2-180f5ca46161 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:11.687155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fd7c4-b0b7-45cd-9cc2-180f5ca46161 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:11.783555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 549/1101] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:05:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:05:11.789772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b83493-d845-4a23-9cf0-fa0b34472e1a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:11.890266 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b83493-d845-4a23-9cf0-fa0b34472e1a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:11.925873 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ef98b-0bb0-4414-9ada-2aeaa9ec5f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:11.926665 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ef98b-0bb0-4414-9ada-2aeaa9ec5f9f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:11.927054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ef98b-0bb0-4414-9ada-2aeaa9ec5f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:11.927330 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ef98b-0bb0-4414-9ada-2aeaa9ec5f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:11.937526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ef98b-0bb0-4414-9ada-2aeaa9ec5f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:11.939050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ef98b-0bb0-4414-9ada-2aeaa9ec5f9f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-4a9ef98b-0bb0-4414-9ada-2aeaa9ec5f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:05:11.957386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:11.958489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:11.959381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:05:11.959381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:05:11.959381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:05:11.959381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:11.962007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 553/1102] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:11.964283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fd7c4-b0b7-45cd-9cc2-180f5ca46161 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:11.982110 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:12.064574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fd7c4-b0b7-45cd-9cc2-180f5ca46161 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:12.098728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb26e0d-e45d-4729-a306-30699be2d353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:12.099387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb26e0d-e45d-4729-a306-30699be2d353 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:12.099667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb26e0d-e45d-4729-a306-30699be2d353 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:12.099939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb26e0d-e45d-4729-a306-30699be2d353 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:12.100651 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb26e0d-e45d-4729-a306-30699be2d353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:12.100956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb26e0d-e45d-4729-a306-30699be2d353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:12.102343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb26e0d-e45d-4729-a306-30699be2d353 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:12.113688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 550/1103] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:12.119189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ef98b-0bb0-4414-9ada-2aeaa9ec5f9f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:12.170397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:12.219995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ef98b-0bb0-4414-9ada-2aeaa9ec5f9f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:12.253444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f670923-77c9-4def-8737-46f747b337c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:12.254254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f670923-77c9-4def-8737-46f747b337c2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:12.254535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f670923-77c9-4def-8737-46f747b337c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:12.254805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f670923-77c9-4def-8737-46f747b337c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:12.262234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f670923-77c9-4def-8737-46f747b337c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:12.263696 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f670923-77c9-4def-8737-46f747b337c2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-1f670923-77c9-4def-8737-46f747b337c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:05:12.277663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:12.278645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:12.279526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:05:12.279526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:05:12.279526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:05:12.279526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:05:12.279526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:05:12.279526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:05:12.279526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:05:12.279526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:12.279526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 554/1104] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:05:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:12.284711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb26e0d-e45d-4729-a306-30699be2d353 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:12.372263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:12.395068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb26e0d-e45d-4729-a306-30699be2d353 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:12.428262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774e565-33dc-4fb3-9a93-c9f353929660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:12.428845 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774e565-33dc-4fb3-9a93-c9f353929660 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:12.429125 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774e565-33dc-4fb3-9a93-c9f353929660 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:12.429400 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774e565-33dc-4fb3-9a93-c9f353929660 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:12.430209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774e565-33dc-4fb3-9a93-c9f353929660 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:12.430520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774e565-33dc-4fb3-9a93-c9f353929660 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:12.431950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774e565-33dc-4fb3-9a93-c9f353929660 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:12.444679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 551/1105] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:05:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:12.451428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f670923-77c9-4def-8737-46f747b337c2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:12.562746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f670923-77c9-4def-8737-46f747b337c2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:12.578357 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:12.612620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1fecdd-b5c7-4424-9c4d-229526fa2280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:12.613263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fecdd-b5c7-4424-9c4d-229526fa2280 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:12.613542 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fecdd-b5c7-4424-9c4d-229526fa2280 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:12.613845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fecdd-b5c7-4424-9c4d-229526fa2280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:12.614665 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1fecdd-b5c7-4424-9c4d-229526fa2280 None admin] RBAC: Adding query filter params (role.id=e9715685617b47c7809732b0a8c61c35, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=d420e808730d484f80963322c5fc6856) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:12.614973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1fecdd-b5c7-4424-9c4d-229526fa2280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:12.616504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1fecdd-b5c7-4424-9c4d-229526fa2280 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:12.623626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 555/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:05:12 2022] GET /identity/v3/role_assignments?user.id=d420e808730d484f80963322c5fc6856&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=e9715685617b47c7809732b0a8c61c35 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:13.717384 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4ec98-b6a6-4ab2-935b-d4790ad0c08f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:13.718005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4ec98-b6a6-4ab2-935b-d4790ad0c08f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:13.718005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4ec98-b6a6-4ab2-935b-d4790ad0c08f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:13.719323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 552/1107] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:13.724594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f11eb-861f-47d2-a101-312552b58479 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:13.725005 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f11eb-861f-47d2-a101-312552b58479 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:13.725139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f11eb-861f-47d2-a101-312552b58479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:13.742447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-934f11eb-861f-47d2-a101-312552b58479 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:13.743627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-934f11eb-861f-47d2-a101-312552b58479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:13.809057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 556/1108] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:13.833676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4ec98-b6a6-4ab2-935b-d4790ad0c08f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:13.937537 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4ec98-b6a6-4ab2-935b-d4790ad0c08f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:13.970620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999a158-daa3-4185-996c-2b94a577c0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:13.971319 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999a158-daa3-4185-996c-2b94a577c0aa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:13.971599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999a158-daa3-4185-996c-2b94a577c0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:13.971865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999a158-daa3-4185-996c-2b94a577c0aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:13.973003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999a158-daa3-4185-996c-2b94a577c0aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:13.974445 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999a158-daa3-4185-996c-2b94a577c0aa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:13.976036 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 553/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:13.982257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f11eb-861f-47d2-a101-312552b58479 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:14.080304 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f11eb-861f-47d2-a101-312552b58479 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:14.082672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:14.115163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4eee3-8613-4b13-86de-379f87c13923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:14.115823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4eee3-8613-4b13-86de-379f87c13923 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:14.116101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4eee3-8613-4b13-86de-379f87c13923 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:14.116370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4eee3-8613-4b13-86de-379f87c13923 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:14.117079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4eee3-8613-4b13-86de-379f87c13923 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:14.118564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4eee3-8613-4b13-86de-379f87c13923 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:14.142294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 557/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:05:13 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:05:16.448849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152abb-3d4a-4247-97a8-3660e5f68b25 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:16.449413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152abb-3d4a-4247-97a8-3660e5f68b25 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:16.449413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152abb-3d4a-4247-97a8-3660e5f68b25 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:16.450810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 554/1111] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:16.456068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977e6af-439a-428d-9e79-2a87f643b06a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:16.456443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977e6af-439a-428d-9e79-2a87f643b06a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:16.456607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977e6af-439a-428d-9e79-2a87f643b06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:16.473548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e977e6af-439a-428d-9e79-2a87f643b06a None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:16.474728 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e977e6af-439a-428d-9e79-2a87f643b06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:16.539411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 558/1112] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:16.563991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152abb-3d4a-4247-97a8-3660e5f68b25 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:16.567703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:16.662943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152abb-3d4a-4247-97a8-3660e5f68b25 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:16.696821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af7f4f-c8a3-4b4b-96df-565076d33127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:16.697702 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af7f4f-c8a3-4b4b-96df-565076d33127 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:16.698159 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af7f4f-c8a3-4b4b-96df-565076d33127 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:16.698556 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af7f4f-c8a3-4b4b-96df-565076d33127 None admin] PATH_INFO: `/v3/roles/506b3cf33324478abe0f72a9d408aa54` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:16.699987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af7f4f-c8a3-4b4b-96df-565076d33127 None admin] RBAC: Authorizing `identity:get_role(role_id=506b3cf33324478abe0f72a9d408aa54)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:16.701334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af7f4f-c8a3-4b4b-96df-565076d33127 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:16.702721 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 555/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:16 2022] GET /identity/v3/roles/506b3cf33324478abe0f72a9d408aa54 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:16.708426 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977e6af-439a-428d-9e79-2a87f643b06a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:16.797747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:16.808085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977e6af-439a-428d-9e79-2a87f643b06a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:16.841391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ebac0-e222-4c99-900e-7d3643495321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:16.842266 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ebac0-e222-4c99-900e-7d3643495321 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:16.842546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ebac0-e222-4c99-900e-7d3643495321 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:16.842815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ebac0-e222-4c99-900e-7d3643495321 None admin] PATH_INFO: `/v3/users/32e2c8b331b745baaf5cb89437e05d74` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:16.858302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ebac0-e222-4c99-900e-7d3643495321 None admin] RBAC: Authorizing `identity:get_user(user_id=32e2c8b331b745baaf5cb89437e05d74)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:16.859777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ebac0-e222-4c99-900e-7d3643495321 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:16.861417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 559/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:16 2022] GET /identity/v3/users/32e2c8b331b745baaf5cb89437e05d74 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:16.867074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af7f4f-c8a3-4b4b-96df-565076d33127 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:16.940100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:16.970538 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af7f4f-c8a3-4b4b-96df-565076d33127 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:17.006939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1e619-3048-43b6-8aeb-b2ddcccc0d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:17.007585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1e619-3048-43b6-8aeb-b2ddcccc0d96 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:17.007866 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1e619-3048-43b6-8aeb-b2ddcccc0d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:17.008139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1e619-3048-43b6-8aeb-b2ddcccc0d96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:17.077880 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1e619-3048-43b6-8aeb-b2ddcccc0d96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:17.079331 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1e619-3048-43b6-8aeb-b2ddcccc0d96 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:17.147322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:17.179589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 556/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:05:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:05:17.185411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ebac0-e222-4c99-900e-7d3643495321 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:17.286384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ebac0-e222-4c99-900e-7d3643495321 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:17.321940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202c66b-636c-4644-a583-abddb5c76f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:17.322821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202c66b-636c-4644-a583-abddb5c76f22 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:17.323101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202c66b-636c-4644-a583-abddb5c76f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:17.323372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202c66b-636c-4644-a583-abddb5c76f22 None admin] PATH_INFO: `/v3/projects/3547af6a2ae6426baefee43df7476d2b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:17.324543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202c66b-636c-4644-a583-abddb5c76f22 None admin] RBAC: Authorizing `identity:get_project(project_id=3547af6a2ae6426baefee43df7476d2b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:17.326029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202c66b-636c-4644-a583-abddb5c76f22 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:17.327759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 560/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:05:17 2022] GET /identity/v3/projects/3547af6a2ae6426baefee43df7476d2b => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:17.333720 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1e619-3048-43b6-8aeb-b2ddcccc0d96 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:17.435176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1e619-3048-43b6-8aeb-b2ddcccc0d96 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:17.493286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc82bf-8737-4174-a7ce-fec8aa16be87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:17.494425 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc82bf-8737-4174-a7ce-fec8aa16be87 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:17.494920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc82bf-8737-4174-a7ce-fec8aa16be87 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:17.495412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc82bf-8737-4174-a7ce-fec8aa16be87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:17.496950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc82bf-8737-4174-a7ce-fec8aa16be87 None admin] RBAC: Adding query filter params (role.id=506b3cf33324478abe0f72a9d408aa54, scope.project.id=3547af6a2ae6426baefee43df7476d2b, user.id=32e2c8b331b745baaf5cb89437e05d74) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:17.497526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc82bf-8737-4174-a7ce-fec8aa16be87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:17.500303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc82bf-8737-4174-a7ce-fec8aa16be87 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:17.509762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 557/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:05:17 2022] GET /identity/v3/role_assignments?user.id=32e2c8b331b745baaf5cb89437e05d74&scope.project.id=3547af6a2ae6426baefee43df7476d2b&role.id=506b3cf33324478abe0f72a9d408aa54 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:19.784658 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dec5d5-4e01-4b77-9a9d-4a40d2a82d06 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:19.785223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dec5d5-4e01-4b77-9a9d-4a40d2a82d06 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:19.785223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dec5d5-4e01-4b77-9a9d-4a40d2a82d06 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:19.786600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 561/1118] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:19.792475 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbf269-3621-42c3-94b4-b1b1198a13c6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:19.792959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbf269-3621-42c3-94b4-b1b1198a13c6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:19.793073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbf269-3621-42c3-94b4-b1b1198a13c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:19.813072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9cfbf269-3621-42c3-94b4-b1b1198a13c6 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:19.814288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9cfbf269-3621-42c3-94b4-b1b1198a13c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:19.844769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:19.886476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 558/1119] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:19.909308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dec5d5-4e01-4b77-9a9d-4a40d2a82d06 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:19.928275 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:20.018933 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dec5d5-4e01-4b77-9a9d-4a40d2a82d06 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:20.058054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a1a23-2895-4267-b2cc-296dd52f9859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:20.058983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a1a23-2895-4267-b2cc-296dd52f9859 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:20.059267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a1a23-2895-4267-b2cc-296dd52f9859 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:20.059544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a1a23-2895-4267-b2cc-296dd52f9859 None admin] PATH_INFO: `/v3/roles/506b3cf33324478abe0f72a9d408aa54` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:20.061036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a1a23-2895-4267-b2cc-296dd52f9859 None admin] RBAC: Authorizing `identity:get_role(role_id=506b3cf33324478abe0f72a9d408aa54)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:20.062470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a1a23-2895-4267-b2cc-296dd52f9859 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:20.063862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 562/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:19 2022] GET /identity/v3/roles/506b3cf33324478abe0f72a9d408aa54 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:20.070006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbf269-3621-42c3-94b4-b1b1198a13c6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:20.179037 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbf269-3621-42c3-94b4-b1b1198a13c6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:20.218150 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039d8ed-fb32-4ea1-a040-e6ab50440604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:20.219016 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039d8ed-fb32-4ea1-a040-e6ab50440604 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:20.219299 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039d8ed-fb32-4ea1-a040-e6ab50440604 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:20.219573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039d8ed-fb32-4ea1-a040-e6ab50440604 None admin] PATH_INFO: `/v3/users/32e2c8b331b745baaf5cb89437e05d74` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:20.220769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039d8ed-fb32-4ea1-a040-e6ab50440604 None admin] RBAC: Authorizing `identity:get_user(user_id=32e2c8b331b745baaf5cb89437e05d74)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:20.222254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039d8ed-fb32-4ea1-a040-e6ab50440604 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:20.223920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 559/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:20 2022] GET /identity/v3/users/32e2c8b331b745baaf5cb89437e05d74 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:20.231922 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a1a23-2895-4267-b2cc-296dd52f9859 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:20.320068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:20.337069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a1a23-2895-4267-b2cc-296dd52f9859 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:20.378606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff1982-74bb-4be0-9374-899d3b2f23b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:20.379452 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff1982-74bb-4be0-9374-899d3b2f23b6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:20.379724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff1982-74bb-4be0-9374-899d3b2f23b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:20.379995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff1982-74bb-4be0-9374-899d3b2f23b6 None admin] PATH_INFO: `/v3/projects/3547af6a2ae6426baefee43df7476d2b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:20.381170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff1982-74bb-4be0-9374-899d3b2f23b6 None admin] RBAC: Authorizing `identity:get_project(project_id=3547af6a2ae6426baefee43df7476d2b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:20.382647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff1982-74bb-4be0-9374-899d3b2f23b6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:20.384324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 563/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:05:20 2022] GET /identity/v3/projects/3547af6a2ae6426baefee43df7476d2b => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:20.390607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039d8ed-fb32-4ea1-a040-e6ab50440604 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:20.427857 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:20.499361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039d8ed-fb32-4ea1-a040-e6ab50440604 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:20.539048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e42fc4-afcb-47b0-86b0-f3348f806ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:20.539931 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e42fc4-afcb-47b0-86b0-f3348f806ad5 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:20.540216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e42fc4-afcb-47b0-86b0-f3348f806ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:20.540488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e42fc4-afcb-47b0-86b0-f3348f806ad5 None admin] PATH_INFO: `/v3/projects/3547af6a2ae6426baefee43df7476d2b/users/32e2c8b331b745baaf5cb89437e05d74/roles/506b3cf33324478abe0f72a9d408aa54` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:20.543351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e42fc4-afcb-47b0-86b0-f3348f806ad5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3547af6a2ae6426baefee43df7476d2b, user_id=32e2c8b331b745baaf5cb89437e05d74, role_id=506b3cf33324478abe0f72a9d408aa54)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:20.544791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e42fc4-afcb-47b0-86b0-f3348f806ad5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:20.560379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 560/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:05:20 2022] PUT /identity/v3/projects/3547af6a2ae6426baefee43df7476d2b/users/32e2c8b331b745baaf5cb89437e05d74/roles/506b3cf33324478abe0f72a9d408aa54 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:22.798365 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e09e38-d50b-4c47-855a-34e9f1df5115 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:22.798928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e09e38-d50b-4c47-855a-34e9f1df5115 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:22.798928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e09e38-d50b-4c47-855a-34e9f1df5115 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:22.800274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 564/1124] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:22.805623 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfca87-8c4c-457e-a9df-1fd5fa338f23 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:22.806094 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfca87-8c4c-457e-a9df-1fd5fa338f23 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:22.806226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfca87-8c4c-457e-a9df-1fd5fa338f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:22.824905 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-fedfca87-8c4c-457e-a9df-1fd5fa338f23 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:22.826146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-fedfca87-8c4c-457e-a9df-1fd5fa338f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:22.895361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 561/1125] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:22.920241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e09e38-d50b-4c47-855a-34e9f1df5115 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.031080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e09e38-d50b-4c47-855a-34e9f1df5115 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.066758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9aed9-9101-4a6c-b45e-5e3934cee014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.067533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9aed9-9101-4a6c-b45e-5e3934cee014 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.067811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9aed9-9101-4a6c-b45e-5e3934cee014 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.068081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9aed9-9101-4a6c-b45e-5e3934cee014 None admin] PATH_INFO: `/v3/roles/506b3cf33324478abe0f72a9d408aa54` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.069540 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9aed9-9101-4a6c-b45e-5e3934cee014 None admin] RBAC: Authorizing `identity:get_role(role_id=506b3cf33324478abe0f72a9d408aa54)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.070977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9aed9-9101-4a6c-b45e-5e3934cee014 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.072355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 565/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:22 2022] GET /identity/v3/roles/506b3cf33324478abe0f72a9d408aa54 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:23.079081 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfca87-8c4c-457e-a9df-1fd5fa338f23 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.104543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:23.187124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfca87-8c4c-457e-a9df-1fd5fa338f23 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.221212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6d0e7-f96d-4977-970c-090d731f9994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.222111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6d0e7-f96d-4977-970c-090d731f9994 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.222390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6d0e7-f96d-4977-970c-090d731f9994 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.222662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6d0e7-f96d-4977-970c-090d731f9994 None admin] PATH_INFO: `/v3/users/32e2c8b331b745baaf5cb89437e05d74` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.223877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6d0e7-f96d-4977-970c-090d731f9994 None admin] RBAC: Authorizing `identity:get_user(user_id=32e2c8b331b745baaf5cb89437e05d74)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.225319 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6d0e7-f96d-4977-970c-090d731f9994 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.227016 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 562/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:23 2022] GET /identity/v3/users/32e2c8b331b745baaf5cb89437e05d74 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:23.233340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9aed9-9101-4a6c-b45e-5e3934cee014 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.256675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:23.346192 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9aed9-9101-4a6c-b45e-5e3934cee014 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.382373 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb68d3e-19e8-41b6-ac22-22e8eeba54f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.383023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb68d3e-19e8-41b6-ac22-22e8eeba54f0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.383306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb68d3e-19e8-41b6-ac22-22e8eeba54f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.383578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb68d3e-19e8-41b6-ac22-22e8eeba54f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.458260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb68d3e-19e8-41b6-ac22-22e8eeba54f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.459756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb68d3e-19e8-41b6-ac22-22e8eeba54f0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.481658 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:23.565048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 566/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:05:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:05:23.571059 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6d0e7-f96d-4977-970c-090d731f9994 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.667169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:23.676000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6d0e7-f96d-4977-970c-090d731f9994 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.715207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31ea59-dc41-4286-9187-23959590edb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.716299 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31ea59-dc41-4286-9187-23959590edb9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.716582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31ea59-dc41-4286-9187-23959590edb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.716935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31ea59-dc41-4286-9187-23959590edb9 None admin] PATH_INFO: `/v3/projects/3547af6a2ae6426baefee43df7476d2b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.718288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31ea59-dc41-4286-9187-23959590edb9 None admin] RBAC: Authorizing `identity:get_project(project_id=3547af6a2ae6426baefee43df7476d2b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.720220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31ea59-dc41-4286-9187-23959590edb9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.722648 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 563/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:05:23 2022] GET /identity/v3/projects/3547af6a2ae6426baefee43df7476d2b => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:23.728968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb68d3e-19e8-41b6-ac22-22e8eeba54f0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.845522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb68d3e-19e8-41b6-ac22-22e8eeba54f0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.867369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:23.882738 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf38b6-c28b-4a84-b8c8-5f47b655b706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.883387 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf38b6-c28b-4a84-b8c8-5f47b655b706 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.883663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf38b6-c28b-4a84-b8c8-5f47b655b706 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.883931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf38b6-c28b-4a84-b8c8-5f47b655b706 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.884753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf38b6-c28b-4a84-b8c8-5f47b655b706 None admin] RBAC: Adding query filter params (role.id=506b3cf33324478abe0f72a9d408aa54, scope.project.id=3547af6a2ae6426baefee43df7476d2b, user.id=32e2c8b331b745baaf5cb89437e05d74) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:23.885093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf38b6-c28b-4a84-b8c8-5f47b655b706 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.886950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf38b6-c28b-4a84-b8c8-5f47b655b706 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.893099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 567/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:05:23 2022] GET /identity/v3/role_assignments?user.id=32e2c8b331b745baaf5cb89437e05d74&scope.project.id=3547af6a2ae6426baefee43df7476d2b&role.id=506b3cf33324478abe0f72a9d408aa54 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:26.165802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737f0c4-75dc-4628-a9b2-7d7627808249 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:26.166896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737f0c4-75dc-4628-a9b2-7d7627808249 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:26.166988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737f0c4-75dc-4628-a9b2-7d7627808249 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:26.168461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 564/1131] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:26.174075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8398e37-22f2-4c13-b896-aedd76191a48 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:26.174585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8398e37-22f2-4c13-b896-aedd76191a48 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:26.174585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8398e37-22f2-4c13-b896-aedd76191a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:26.192544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c8398e37-22f2-4c13-b896-aedd76191a48 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:26.193867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c8398e37-22f2-4c13-b896-aedd76191a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:26.263472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 568/1132] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:26.287232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737f0c4-75dc-4628-a9b2-7d7627808249 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:26.394865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737f0c4-75dc-4628-a9b2-7d7627808249 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:26.429546 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a36c3c-5779-4ce8-8266-f5cbbf62d8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:26.430220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a36c3c-5779-4ce8-8266-f5cbbf62d8ef None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:26.435320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a36c3c-5779-4ce8-8266-f5cbbf62d8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:26.435320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a36c3c-5779-4ce8-8266-f5cbbf62d8ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:26.435320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a36c3c-5779-4ce8-8266-f5cbbf62d8ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:26.435320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a36c3c-5779-4ce8-8266-f5cbbf62d8ef None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:26.435613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 565/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:26.441463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8398e37-22f2-4c13-b896-aedd76191a48 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:26.541782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8398e37-22f2-4c13-b896-aedd76191a48 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:26.556085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:26.579968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3e214-1b9a-4bb2-980b-da090d6377d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:26.582335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3e214-1b9a-4bb2-980b-da090d6377d8 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:26.582335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3e214-1b9a-4bb2-980b-da090d6377d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:26.582335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3e214-1b9a-4bb2-980b-da090d6377d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:26.582335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3e214-1b9a-4bb2-980b-da090d6377d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:26.583382 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3e214-1b9a-4bb2-980b-da090d6377d8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:26.617719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 569/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:05:26 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:28.893786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838fe1f-7457-4c99-b1ab-3e0d2a7e8e69 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:28.894378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838fe1f-7457-4c99-b1ab-3e0d2a7e8e69 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:28.894378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838fe1f-7457-4c99-b1ab-3e0d2a7e8e69 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:28.895691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 566/1135] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:28.901091 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788558cc-86ba-48d0-a105-dd272bf80726 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:28.901482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788558cc-86ba-48d0-a105-dd272bf80726 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:28.901646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788558cc-86ba-48d0-a105-dd272bf80726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:28.919251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-788558cc-86ba-48d0-a105-dd272bf80726 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:28.920457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-788558cc-86ba-48d0-a105-dd272bf80726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:28.987722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 570/1136] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:29.013800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838fe1f-7457-4c99-b1ab-3e0d2a7e8e69 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:29.041244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:29.113160 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838fe1f-7457-4c99-b1ab-3e0d2a7e8e69 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:29.146318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9d255-411b-40b3-b11f-4488ce4b0107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:29.147092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9d255-411b-40b3-b11f-4488ce4b0107 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:29.147373 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9d255-411b-40b3-b11f-4488ce4b0107 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:29.147646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9d255-411b-40b3-b11f-4488ce4b0107 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:29.148807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9d255-411b-40b3-b11f-4488ce4b0107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:29.150266 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9d255-411b-40b3-b11f-4488ce4b0107 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:29.152113 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 567/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:05:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:29.158023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788558cc-86ba-48d0-a105-dd272bf80726 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:29.250408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:29.256655 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788558cc-86ba-48d0-a105-dd272bf80726 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:29.290609 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b862f9-0307-4881-a19a-0d03690f9977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:29.291794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b862f9-0307-4881-a19a-0d03690f9977 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:29.292074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b862f9-0307-4881-a19a-0d03690f9977 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:29.292343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b862f9-0307-4881-a19a-0d03690f9977 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:29.293037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b862f9-0307-4881-a19a-0d03690f9977 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:29.294534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b862f9-0307-4881-a19a-0d03690f9977 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:29.314370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 571/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:05:29 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:31.564119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901d4e9-cdcf-4763-ad96-ef7a9c9ef017 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.564665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901d4e9-cdcf-4763-ad96-ef7a9c9ef017 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.564665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901d4e9-cdcf-4763-ad96-ef7a9c9ef017 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.566082 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 568/1139] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:31.571302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ad8aa-2dac-46a9-849e-d0f5f9c8f763 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.571729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ad8aa-2dac-46a9-849e-d0f5f9c8f763 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.571831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ad8aa-2dac-46a9-849e-d0f5f9c8f763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.588852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3e7ad8aa-2dac-46a9-849e-d0f5f9c8f763 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:31.590062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3e7ad8aa-2dac-46a9-849e-d0f5f9c8f763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:31.662917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 572/1140] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:31.686750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901d4e9-cdcf-4763-ad96-ef7a9c9ef017 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:31.779975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:31.787506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901d4e9-cdcf-4763-ad96-ef7a9c9ef017 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:31.823373 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f0c3b-5cea-4439-9864-e3813492f8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:31.824069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f0c3b-5cea-4439-9864-e3813492f8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.824350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f0c3b-5cea-4439-9864-e3813492f8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.824626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f0c3b-5cea-4439-9864-e3813492f8a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.834695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f0c3b-5cea-4439-9864-e3813492f8a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f17f0c3b-5cea-4439-9864-e3813492f8a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:05:31.842925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:31.844119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:31.845264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 569/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:05:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:31.850423 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ad8aa-2dac-46a9-849e-d0f5f9c8f763 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:31.932622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:31.958643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ad8aa-2dac-46a9-849e-d0f5f9c8f763 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:31.993531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bb06e-47c2-443b-be82-930715f540bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:31.994258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bb06e-47c2-443b-be82-930715f540bd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.994556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bb06e-47c2-443b-be82-930715f540bd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.994828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bb06e-47c2-443b-be82-930715f540bd None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.995567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bb06e-47c2-443b-be82-930715f540bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:31.995877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bb06e-47c2-443b-be82-930715f540bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:31.997263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bb06e-47c2-443b-be82-930715f540bd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:32.005276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 573/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:05:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:32.011169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f0c3b-5cea-4439-9864-e3813492f8a8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:32.115110 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f0c3b-5cea-4439-9864-e3813492f8a8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:32.151240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03c53d-fb60-40ca-952d-e04cf62d0a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:32.152109 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03c53d-fb60-40ca-952d-e04cf62d0a6b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:32.152387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03c53d-fb60-40ca-952d-e04cf62d0a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:32.152655 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03c53d-fb60-40ca-952d-e04cf62d0a6b None admin] PATH_INFO: `/v3/users/b30910c9bfd44acdad2f78bce1395961` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:32.166925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03c53d-fb60-40ca-952d-e04cf62d0a6b None admin] RBAC: Authorizing `identity:get_user(user_id=b30910c9bfd44acdad2f78bce1395961)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:32.168374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03c53d-fb60-40ca-952d-e04cf62d0a6b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:32.170154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 570/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:32 2022] GET /identity/v3/users/b30910c9bfd44acdad2f78bce1395961 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:32.177661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bb06e-47c2-443b-be82-930715f540bd None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:32.283179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bb06e-47c2-443b-be82-930715f540bd None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:32.316315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c179e1-16d2-49bc-b82b-52b18d2d91dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:32.316954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c179e1-16d2-49bc-b82b-52b18d2d91dc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:32.317236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c179e1-16d2-49bc-b82b-52b18d2d91dc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:32.317511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c179e1-16d2-49bc-b82b-52b18d2d91dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:32.333431 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:32.386349 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c179e1-16d2-49bc-b82b-52b18d2d91dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:32.387727 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c179e1-16d2-49bc-b82b-52b18d2d91dc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:32.485246 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 574/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:05:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:05:32.491173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03c53d-fb60-40ca-952d-e04cf62d0a6b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:32.507991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:32.596621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03c53d-fb60-40ca-952d-e04cf62d0a6b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:32.634045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06edd0-216d-4915-9ef1-1f54fb8d795f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:32.634894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06edd0-216d-4915-9ef1-1f54fb8d795f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:32.635170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06edd0-216d-4915-9ef1-1f54fb8d795f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:32.635442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06edd0-216d-4915-9ef1-1f54fb8d795f None admin] PATH_INFO: `/v3/projects/04b789b62cd34ecf94b49e6727ae3202` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:32.637934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06edd0-216d-4915-9ef1-1f54fb8d795f None admin] RBAC: Authorizing `identity:get_project(project_id=04b789b62cd34ecf94b49e6727ae3202)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:32.639317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06edd0-216d-4915-9ef1-1f54fb8d795f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:32.641039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 571/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:05:32 2022] GET /identity/v3/projects/04b789b62cd34ecf94b49e6727ae3202 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:32.647147 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c179e1-16d2-49bc-b82b-52b18d2d91dc None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:32.694358 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:32.749619 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c179e1-16d2-49bc-b82b-52b18d2d91dc None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:32.784482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3c77f-aead-4cad-bf79-8eb2372bf118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:32.785128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3c77f-aead-4cad-bf79-8eb2372bf118 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:32.785409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3c77f-aead-4cad-bf79-8eb2372bf118 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:32.785677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3c77f-aead-4cad-bf79-8eb2372bf118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:32.786523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3c77f-aead-4cad-bf79-8eb2372bf118 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=04b789b62cd34ecf94b49e6727ae3202, user.id=b30910c9bfd44acdad2f78bce1395961) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:32.786833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3c77f-aead-4cad-bf79-8eb2372bf118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:32.788541 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3c77f-aead-4cad-bf79-8eb2372bf118 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:32.794143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 575/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:05:32 2022] GET /identity/v3/role_assignments?user.id=b30910c9bfd44acdad2f78bce1395961&scope.project.id=04b789b62cd34ecf94b49e6727ae3202&role.id=584beecb022b422f820e77f7623ada5e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:35.011484 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280dc79-28e7-418d-93b9-df59d94005af None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:35.012018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280dc79-28e7-418d-93b9-df59d94005af None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:35.012018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280dc79-28e7-418d-93b9-df59d94005af None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:35.014029 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 572/1147] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:35.019219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245f4a3-44b9-4176-9280-24e4187d8f72 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:35.019649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245f4a3-44b9-4176-9280-24e4187d8f72 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:35.019770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245f4a3-44b9-4176-9280-24e4187d8f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:35.036528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8245f4a3-44b9-4176-9280-24e4187d8f72 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:35.037999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8245f4a3-44b9-4176-9280-24e4187d8f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:35.106745 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 576/1148] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:05:35.129851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280dc79-28e7-418d-93b9-df59d94005af None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:35.216086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:35.237299 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280dc79-28e7-418d-93b9-df59d94005af None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:35.273141 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9433a-c8a4-4d17-b9e7-2cbfe3e66a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:35.273873 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9433a-c8a4-4d17-b9e7-2cbfe3e66a55 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:35.274158 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9433a-c8a4-4d17-b9e7-2cbfe3e66a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:35.274430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9433a-c8a4-4d17-b9e7-2cbfe3e66a55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:35.284897 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9433a-c8a4-4d17-b9e7-2cbfe3e66a55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-46f9433a-c8a4-4d17-b9e7-2cbfe3e66a55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:05:35.293352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:35.294658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:35.296010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 573/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:05:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:05:35.300862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245f4a3-44b9-4176-9280-24e4187d8f72 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:35.341881 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:35.411605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245f4a3-44b9-4176-9280-24e4187d8f72 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:35.451518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020f1c3-04b4-4fd9-ae10-6368a03fcf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:35.452684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020f1c3-04b4-4fd9-ae10-6368a03fcf53 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:35.453188 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020f1c3-04b4-4fd9-ae10-6368a03fcf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:35.453680 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020f1c3-04b4-4fd9-ae10-6368a03fcf53 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:35.455100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020f1c3-04b4-4fd9-ae10-6368a03fcf53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:35.455632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020f1c3-04b4-4fd9-ae10-6368a03fcf53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:35.458122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020f1c3-04b4-4fd9-ae10-6368a03fcf53 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:35.471803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 577/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:05:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:35.478013 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9433a-c8a4-4d17-b9e7-2cbfe3e66a55 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:35.585236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9433a-c8a4-4d17-b9e7-2cbfe3e66a55 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:35.618981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae96e8-6d6e-4cb7-97b6-e193a3b33505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:35.620265 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae96e8-6d6e-4cb7-97b6-e193a3b33505 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:35.620618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae96e8-6d6e-4cb7-97b6-e193a3b33505 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:35.620833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae96e8-6d6e-4cb7-97b6-e193a3b33505 None admin] PATH_INFO: `/v3/users/b30910c9bfd44acdad2f78bce1395961` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:35.622073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae96e8-6d6e-4cb7-97b6-e193a3b33505 None admin] RBAC: Authorizing `identity:get_user(user_id=b30910c9bfd44acdad2f78bce1395961)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:35.623504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae96e8-6d6e-4cb7-97b6-e193a3b33505 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:35.625157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 574/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:35 2022] GET /identity/v3/users/b30910c9bfd44acdad2f78bce1395961 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:35.630877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020f1c3-04b4-4fd9-ae10-6368a03fcf53 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:35.732204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:35.735211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020f1c3-04b4-4fd9-ae10-6368a03fcf53 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:35.768982 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07effd2-8046-4199-8f7c-a2394081c351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:35.769874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07effd2-8046-4199-8f7c-a2394081c351 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:35.770161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07effd2-8046-4199-8f7c-a2394081c351 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:35.770444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07effd2-8046-4199-8f7c-a2394081c351 None admin] PATH_INFO: `/v3/projects/04b789b62cd34ecf94b49e6727ae3202` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:35.771623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07effd2-8046-4199-8f7c-a2394081c351 None admin] RBAC: Authorizing `identity:get_project(project_id=04b789b62cd34ecf94b49e6727ae3202)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:35.773085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07effd2-8046-4199-8f7c-a2394081c351 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:35.774826 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 578/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:05:35 2022] GET /identity/v3/projects/04b789b62cd34ecf94b49e6727ae3202 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:35.781473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae96e8-6d6e-4cb7-97b6-e193a3b33505 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:35.795431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:35.885106 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae96e8-6d6e-4cb7-97b6-e193a3b33505 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:35.920956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02512b05-786f-43e0-8455-516b4c677206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:35.921877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02512b05-786f-43e0-8455-516b4c677206 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:35.922158 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02512b05-786f-43e0-8455-516b4c677206 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:35.922431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02512b05-786f-43e0-8455-516b4c677206 None admin] PATH_INFO: `/v3/projects/04b789b62cd34ecf94b49e6727ae3202/users/b30910c9bfd44acdad2f78bce1395961/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:35.924391 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02512b05-786f-43e0-8455-516b4c677206 None admin] RBAC: Authorizing `identity:create_grant(project_id=04b789b62cd34ecf94b49e6727ae3202, user_id=b30910c9bfd44acdad2f78bce1395961, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:35.925842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02512b05-786f-43e0-8455-516b4c677206 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:35.941343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 575/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:05:35 2022] PUT /identity/v3/projects/04b789b62cd34ecf94b49e6727ae3202/users/b30910c9bfd44acdad2f78bce1395961/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:38.233916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94192aa6-61a1-41b0-bf79-1ce18619433f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:38.234468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94192aa6-61a1-41b0-bf79-1ce18619433f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:38.234468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94192aa6-61a1-41b0-bf79-1ce18619433f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:38.235826 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 579/1154] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:38.241135 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1f3a0-403f-4302-9bdf-f62037834391 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:38.241501 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1f3a0-403f-4302-9bdf-f62037834391 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:38.241682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1f3a0-403f-4302-9bdf-f62037834391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:38.258493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c6a1f3a0-403f-4302-9bdf-f62037834391 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:38.259645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c6a1f3a0-403f-4302-9bdf-f62037834391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:38.327171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:38.332741 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 576/1155] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:38.357025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94192aa6-61a1-41b0-bf79-1ce18619433f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:38.458210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94192aa6-61a1-41b0-bf79-1ce18619433f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:38.490215 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dad748-a851-429a-bd98-c11e4cf1014c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:38.490933 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dad748-a851-429a-bd98-c11e4cf1014c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:38.491212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dad748-a851-429a-bd98-c11e4cf1014c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:38.491482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dad748-a851-429a-bd98-c11e4cf1014c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:38.501907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dad748-a851-429a-bd98-c11e4cf1014c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-10dad748-a851-429a-bd98-c11e4cf1014c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:05:38.509451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:38.511361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:38.512429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 580/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:05:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:38.517474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1f3a0-403f-4302-9bdf-f62037834391 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:38.613474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1f3a0-403f-4302-9bdf-f62037834391 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:38.647139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d6efa-daef-4c22-8ef5-337a263bb171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:38.647794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d6efa-daef-4c22-8ef5-337a263bb171 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:38.648074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d6efa-daef-4c22-8ef5-337a263bb171 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:38.648349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d6efa-daef-4c22-8ef5-337a263bb171 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:38.649077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d6efa-daef-4c22-8ef5-337a263bb171 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:38.649382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d6efa-daef-4c22-8ef5-337a263bb171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:38.650767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d6efa-daef-4c22-8ef5-337a263bb171 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:38.658621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 577/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:05:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:38.664704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dad748-a851-429a-bd98-c11e4cf1014c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:38.693124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:38.763447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dad748-a851-429a-bd98-c11e4cf1014c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:38.796044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e7274-6809-4a93-8301-d07a38f7832e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:38.796899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e7274-6809-4a93-8301-d07a38f7832e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:38.797175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e7274-6809-4a93-8301-d07a38f7832e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:38.797440 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e7274-6809-4a93-8301-d07a38f7832e None admin] PATH_INFO: `/v3/users/b30910c9bfd44acdad2f78bce1395961` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:38.798635 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e7274-6809-4a93-8301-d07a38f7832e None admin] RBAC: Authorizing `identity:get_user(user_id=b30910c9bfd44acdad2f78bce1395961)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:38.800080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e7274-6809-4a93-8301-d07a38f7832e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:38.801679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 581/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:38 2022] GET /identity/v3/users/b30910c9bfd44acdad2f78bce1395961 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:38.808093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d6efa-daef-4c22-8ef5-337a263bb171 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:38.868631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:38.910559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d6efa-daef-4c22-8ef5-337a263bb171 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:38.953131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7a755-18e4-4595-b172-542963df2813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:38.953794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7a755-18e4-4595-b172-542963df2813 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:38.954076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7a755-18e4-4595-b172-542963df2813 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:38.954345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7a755-18e4-4595-b172-542963df2813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:39.028687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7a755-18e4-4595-b172-542963df2813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:39.030034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7a755-18e4-4595-b172-542963df2813 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:39.097149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:39.145645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 578/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:05:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:05:39.151737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e7274-6809-4a93-8301-d07a38f7832e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:39.251784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:39.258283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e7274-6809-4a93-8301-d07a38f7832e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:39.297627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb98a7-62eb-4368-81c4-d06c022af560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:39.298449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb98a7-62eb-4368-81c4-d06c022af560 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:39.298729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb98a7-62eb-4368-81c4-d06c022af560 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:39.298999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb98a7-62eb-4368-81c4-d06c022af560 None admin] PATH_INFO: `/v3/projects/04b789b62cd34ecf94b49e6727ae3202` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:39.300156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb98a7-62eb-4368-81c4-d06c022af560 None admin] RBAC: Authorizing `identity:get_project(project_id=04b789b62cd34ecf94b49e6727ae3202)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:39.301639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb98a7-62eb-4368-81c4-d06c022af560 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:39.303361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 582/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:05:39 2022] GET /identity/v3/projects/04b789b62cd34ecf94b49e6727ae3202 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:39.309492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7a755-18e4-4595-b172-542963df2813 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:39.426216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7a755-18e4-4595-b172-542963df2813 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:39.461281 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a06da-2c84-4b0f-80f8-4e03c6ee9bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:39.461950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a06da-2c84-4b0f-80f8-4e03c6ee9bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:39.462228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a06da-2c84-4b0f-80f8-4e03c6ee9bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:39.462500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a06da-2c84-4b0f-80f8-4e03c6ee9bb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:39.463316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a06da-2c84-4b0f-80f8-4e03c6ee9bb9 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=04b789b62cd34ecf94b49e6727ae3202, user.id=b30910c9bfd44acdad2f78bce1395961) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:39.464152 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a06da-2c84-4b0f-80f8-4e03c6ee9bb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:39.466009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a06da-2c84-4b0f-80f8-4e03c6ee9bb9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:39.471891 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 579/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:05:39 2022] GET /identity/v3/role_assignments?user.id=b30910c9bfd44acdad2f78bce1395961&scope.project.id=04b789b62cd34ecf94b49e6727ae3202&role.id=584beecb022b422f820e77f7623ada5e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:41.691571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7aafc-3dec-4893-ae33-81f2d9173689 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:41.692114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7aafc-3dec-4893-ae33-81f2d9173689 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:41.692114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7aafc-3dec-4893-ae33-81f2d9173689 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:41.693467 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 583/1162] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:41.699092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff6167-9985-481f-bc74-a955a671c12e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:41.699517 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff6167-9985-481f-bc74-a955a671c12e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:41.699624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff6167-9985-481f-bc74-a955a671c12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:41.716222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5cff6167-9985-481f-bc74-a955a671c12e None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:41.717385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5cff6167-9985-481f-bc74-a955a671c12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:41.737112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:41.782544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 580/1163] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:41.804889 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7aafc-3dec-4893-ae33-81f2d9173689 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:41.906332 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7aafc-3dec-4893-ae33-81f2d9173689 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:41.947968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1be5c-41bf-41f3-a7ec-0e272a9acdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:41.949427 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1be5c-41bf-41f3-a7ec-0e272a9acdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:41.949782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1be5c-41bf-41f3-a7ec-0e272a9acdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:41.950051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1be5c-41bf-41f3-a7ec-0e272a9acdf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:41.981603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:42.022478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1be5c-41bf-41f3-a7ec-0e272a9acdf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:42.023947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1be5c-41bf-41f3-a7ec-0e272a9acdf3 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:42.129542 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 584/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:05:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:05:42.136281 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff6167-9985-481f-bc74-a955a671c12e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:42.241043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff6167-9985-481f-bc74-a955a671c12e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:42.277053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef82726-606d-47d8-bcf1-f5556aa1b413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:42.277890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef82726-606d-47d8-bcf1-f5556aa1b413 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:42.278169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef82726-606d-47d8-bcf1-f5556aa1b413 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:42.278438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef82726-606d-47d8-bcf1-f5556aa1b413 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:42.285229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef82726-606d-47d8-bcf1-f5556aa1b413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:42.286695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef82726-606d-47d8-bcf1-f5556aa1b413 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-3ef82726-606d-47d8-bcf1-f5556aa1b413 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:42.301413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:42.302722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:42.304042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 581/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 18:05:42 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:42.309234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1be5c-41bf-41f3-a7ec-0e272a9acdf3 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:42.370932 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:42.411926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1be5c-41bf-41f3-a7ec-0e272a9acdf3 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:42.447139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beee7dc-1206-4fc4-a190-7ee7b9f03517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:42.447872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beee7dc-1206-4fc4-a190-7ee7b9f03517 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:42.448240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beee7dc-1206-4fc4-a190-7ee7b9f03517 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:42.448550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beee7dc-1206-4fc4-a190-7ee7b9f03517 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:42.449375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beee7dc-1206-4fc4-a190-7ee7b9f03517 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:42.449683 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beee7dc-1206-4fc4-a190-7ee7b9f03517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:42.451098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beee7dc-1206-4fc4-a190-7ee7b9f03517 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:42.458463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 585/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 18:05:42 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:44.680767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3485388-a472-4be5-ac93-2a049cbb21ee None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.681323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3485388-a472-4be5-ac93-2a049cbb21ee None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.681323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3485388-a472-4be5-ac93-2a049cbb21ee None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:44.682982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 582/1167] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:05:44.688548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7062ff6-3f50-472a-8c8b-def7fabaa567 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.688986 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7062ff6-3f50-472a-8c8b-def7fabaa567 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.689088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7062ff6-3f50-472a-8c8b-def7fabaa567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:44.706404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d7062ff6-3f50-472a-8c8b-def7fabaa567 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:44.707605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d7062ff6-3f50-472a-8c8b-def7fabaa567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:44.774433 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 586/1168] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:44.798507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3485388-a472-4be5-ac93-2a049cbb21ee None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:44.817422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:44.906032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3485388-a472-4be5-ac93-2a049cbb21ee None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:44.940154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2999b46-569e-41cb-b158-25b0068ca19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:44.940727 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2999b46-569e-41cb-b158-25b0068ca19a None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.941009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2999b46-569e-41cb-b158-25b0068ca19a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.941283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2999b46-569e-41cb-b158-25b0068ca19a None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:44.942054 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2999b46-569e-41cb-b158-25b0068ca19a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:44.943422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2999b46-569e-41cb-b158-25b0068ca19a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:44.958216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 583/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Thu May 26 18:05:44 2022] POST /identity/v3/domains => generated 267 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:47.246506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631af14-710e-4a15-a353-237a7d2d6924 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:47.247084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631af14-710e-4a15-a353-237a7d2d6924 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:47.247084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631af14-710e-4a15-a353-237a7d2d6924 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:47.248408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 587/1170] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:47.253665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f5dd8-442f-45eb-8653-36b7c1c62606 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:47.254079 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f5dd8-442f-45eb-8653-36b7c1c62606 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:47.254237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f5dd8-442f-45eb-8653-36b7c1c62606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:47.270775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a53f5dd8-442f-45eb-8653-36b7c1c62606 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:47.271952 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a53f5dd8-442f-45eb-8653-36b7c1c62606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:47.339091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 584/1171] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:47.363407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631af14-710e-4a15-a353-237a7d2d6924 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:47.422773 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:47.465009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631af14-710e-4a15-a353-237a7d2d6924 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:47.498534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d440603-f904-47ca-b351-25da982dd2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:47.499314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d440603-f904-47ca-b351-25da982dd2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:47.499593 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d440603-f904-47ca-b351-25da982dd2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:47.499865 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d440603-f904-47ca-b351-25da982dd2b5 None admin] PATH_INFO: `/v3/domains/24423bcfe8d04f7dad5582c5ea3619cd` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:47.511190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d440603-f904-47ca-b351-25da982dd2b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=24423bcfe8d04f7dad5582c5ea3619cd)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:47.513620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d440603-f904-47ca-b351-25da982dd2b5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:47.515943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 588/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Thu May 26 18:05:47 2022] GET /identity/v3/domains/24423bcfe8d04f7dad5582c5ea3619cd => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:47.522156 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f5dd8-442f-45eb-8653-36b7c1c62606 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:47.531568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:47.622752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f5dd8-442f-45eb-8653-36b7c1c62606 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:47.661408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de7516-3f0f-412b-b764-125ef3ae6955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:47.662109 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de7516-3f0f-412b-b764-125ef3ae6955 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:47.662645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de7516-3f0f-412b-b764-125ef3ae6955 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:47.662891 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de7516-3f0f-412b-b764-125ef3ae6955 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:47.663596 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de7516-3f0f-412b-b764-125ef3ae6955 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:47.665032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de7516-3f0f-412b-b764-125ef3ae6955 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:47.690345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 585/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:05:47 2022] POST /identity/v3/projects => generated 361 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:49.935366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdd272-f7ca-42a8-b25c-9750971d210c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:49.935999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdd272-f7ca-42a8-b25c-9750971d210c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:49.935999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdd272-f7ca-42a8-b25c-9750971d210c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:49.937261 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 589/1174] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:49.942446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfd403-ab52-43dc-94d6-66290b7aef8b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:49.942859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfd403-ab52-43dc-94d6-66290b7aef8b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:49.942998 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfd403-ab52-43dc-94d6-66290b7aef8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:49.959771 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bebfd403-ab52-43dc-94d6-66290b7aef8b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:49.960941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bebfd403-ab52-43dc-94d6-66290b7aef8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:50.014195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:50.028756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 586/1175] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:50.053097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdd272-f7ca-42a8-b25c-9750971d210c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:50.154562 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdd272-f7ca-42a8-b25c-9750971d210c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:50.171454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:50.188728 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f6930-37de-4722-8672-a84e554ee789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:50.189509 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f6930-37de-4722-8672-a84e554ee789 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:50.189835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f6930-37de-4722-8672-a84e554ee789 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:50.190104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f6930-37de-4722-8672-a84e554ee789 None admin] PATH_INFO: `/v3/domains/24423bcfe8d04f7dad5582c5ea3619cd` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:50.191244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f6930-37de-4722-8672-a84e554ee789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=24423bcfe8d04f7dad5582c5ea3619cd)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:50.192673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f6930-37de-4722-8672-a84e554ee789 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:50.194316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 590/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Thu May 26 18:05:50 2022] GET /identity/v3/domains/24423bcfe8d04f7dad5582c5ea3619cd => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:50.204366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfd403-ab52-43dc-94d6-66290b7aef8b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:50.305709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfd403-ab52-43dc-94d6-66290b7aef8b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:50.340656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dd6e1-41ce-4b84-9140-b8fdc7387fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:50.341315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dd6e1-41ce-4b84-9140-b8fdc7387fb0 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:50.341592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dd6e1-41ce-4b84-9140-b8fdc7387fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:50.341896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dd6e1-41ce-4b84-9140-b8fdc7387fb0 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:50.342585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dd6e1-41ce-4b84-9140-b8fdc7387fb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:50.344002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dd6e1-41ce-4b84-9140-b8fdc7387fb0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:50.364611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 587/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:05:50 2022] POST /identity/v3/users => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:52.606797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d0f0-3e50-487a-8e11-293554721569 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:52.607317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d0f0-3e50-487a-8e11-293554721569 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:52.607317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d0f0-3e50-487a-8e11-293554721569 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:52.608623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 591/1178] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:52.613751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3879bb1-3bc7-4b2e-9b5a-e623c0f7ccc7 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:52.614167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3879bb1-3bc7-4b2e-9b5a-e623c0f7ccc7 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:52.614322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3879bb1-3bc7-4b2e-9b5a-e623c0f7ccc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:52.631112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a3879bb1-3bc7-4b2e-9b5a-e623c0f7ccc7 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:52.632271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a3879bb1-3bc7-4b2e-9b5a-e623c0f7ccc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:52.672021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:52.700247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 588/1179] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:52.723584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46d0f0-3e50-487a-8e11-293554721569 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:52.823770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46d0f0-3e50-487a-8e11-293554721569 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:52.857691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d3f6c-447d-4f36-9bdb-921e969f96ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:52.858428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d3f6c-447d-4f36-9bdb-921e969f96ae None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:52.858711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d3f6c-447d-4f36-9bdb-921e969f96ae None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:52.858984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d3f6c-447d-4f36-9bdb-921e969f96ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:52.868939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d3f6c-447d-4f36-9bdb-921e969f96ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-683d3f6c-447d-4f36-9bdb-921e969f96ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:05:52.877492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:52.878603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:52.879460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 592/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:05:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:52.884703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3879bb1-3bc7-4b2e-9b5a-e623c0f7ccc7 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:52.980290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3879bb1-3bc7-4b2e-9b5a-e623c0f7ccc7 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:53.015415 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8feaf-c283-48ef-a89f-11130a0001b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:53.016070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8feaf-c283-48ef-a89f-11130a0001b4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:53.016355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8feaf-c283-48ef-a89f-11130a0001b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:53.016656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8feaf-c283-48ef-a89f-11130a0001b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:53.017397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8feaf-c283-48ef-a89f-11130a0001b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:53.017705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8feaf-c283-48ef-a89f-11130a0001b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:53.019105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8feaf-c283-48ef-a89f-11130a0001b4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:53.027842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 589/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:05:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:53.033661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d3f6c-447d-4f36-9bdb-921e969f96ae None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:53.079971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:53.133661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d3f6c-447d-4f36-9bdb-921e969f96ae None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:53.167090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c553bf-26bb-4dc3-b428-6229a6237029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:53.167954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c553bf-26bb-4dc3-b428-6229a6237029 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:53.168234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c553bf-26bb-4dc3-b428-6229a6237029 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:53.168506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c553bf-26bb-4dc3-b428-6229a6237029 None admin] PATH_INFO: `/v3/users/a87147222a43464cb334ee15e9d3dc07` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:53.181424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c553bf-26bb-4dc3-b428-6229a6237029 None admin] RBAC: Authorizing `identity:get_user(user_id=a87147222a43464cb334ee15e9d3dc07)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:53.183417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c553bf-26bb-4dc3-b428-6229a6237029 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:53.185062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 593/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:53 2022] GET /identity/v3/users/a87147222a43464cb334ee15e9d3dc07 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:53.190798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8feaf-c283-48ef-a89f-11130a0001b4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:53.226098 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:53.290574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8feaf-c283-48ef-a89f-11130a0001b4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:53.328226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde565dd-70ff-4ed6-b5ac-3cc2469644f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:53.328875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde565dd-70ff-4ed6-b5ac-3cc2469644f8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:53.329155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde565dd-70ff-4ed6-b5ac-3cc2469644f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:53.329427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde565dd-70ff-4ed6-b5ac-3cc2469644f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:53.394610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde565dd-70ff-4ed6-b5ac-3cc2469644f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:53.396049 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde565dd-70ff-4ed6-b5ac-3cc2469644f8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:53.428045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:53.498100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 590/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:05:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:05:53.504223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c553bf-26bb-4dc3-b428-6229a6237029 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:53.616875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c553bf-26bb-4dc3-b428-6229a6237029 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:53.619262 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:53.652624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718378f-1d78-4d7e-8d7b-9cef8656d7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:53.653389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718378f-1d78-4d7e-8d7b-9cef8656d7cf None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:53.653669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718378f-1d78-4d7e-8d7b-9cef8656d7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:53.653970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718378f-1d78-4d7e-8d7b-9cef8656d7cf None admin] PATH_INFO: `/v3/projects/f9fefb0e2616494b9164de20d99a2422` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:53.655189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718378f-1d78-4d7e-8d7b-9cef8656d7cf None admin] RBAC: Authorizing `identity:get_project(project_id=f9fefb0e2616494b9164de20d99a2422)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:53.656653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718378f-1d78-4d7e-8d7b-9cef8656d7cf None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:53.658398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 594/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:05:53 2022] GET /identity/v3/projects/f9fefb0e2616494b9164de20d99a2422 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:53.664761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde565dd-70ff-4ed6-b5ac-3cc2469644f8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:53.767335 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde565dd-70ff-4ed6-b5ac-3cc2469644f8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:53.801183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de4fe4-7d04-475c-bee6-808ae1b95e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:53.801861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de4fe4-7d04-475c-bee6-808ae1b95e52 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:53.802146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de4fe4-7d04-475c-bee6-808ae1b95e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:53.802419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de4fe4-7d04-475c-bee6-808ae1b95e52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:53.803242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de4fe4-7d04-475c-bee6-808ae1b95e52 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=f9fefb0e2616494b9164de20d99a2422, user.id=a87147222a43464cb334ee15e9d3dc07) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:53.803552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de4fe4-7d04-475c-bee6-808ae1b95e52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:53.804982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de4fe4-7d04-475c-bee6-808ae1b95e52 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:53.808507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:53.811549 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 591/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:05:53 2022] GET /identity/v3/role_assignments?user.id=a87147222a43464cb334ee15e9d3dc07&scope.project.id=f9fefb0e2616494b9164de20d99a2422&role.id=584beecb022b422f820e77f7623ada5e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:55.996215 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e374fab-72e0-4d39-850f-b2b37ddf7778 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:55.996776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e374fab-72e0-4d39-850f-b2b37ddf7778 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:55.996776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e374fab-72e0-4d39-850f-b2b37ddf7778 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:55.998139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 595/1186] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:05:56.004076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba5b0a-14ce-4507-88a6-63cb00d158b3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:56.004611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba5b0a-14ce-4507-88a6-63cb00d158b3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:56.004611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba5b0a-14ce-4507-88a6-63cb00d158b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:56.022380 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-74ba5b0a-14ce-4507-88a6-63cb00d158b3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:56.023547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-74ba5b0a-14ce-4507-88a6-63cb00d158b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:56.089920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 592/1187] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:56.112065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e374fab-72e0-4d39-850f-b2b37ddf7778 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:56.212868 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e374fab-72e0-4d39-850f-b2b37ddf7778 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:56.245891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43232e61-5925-4fcf-9ce6-a3092725958f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:56.246716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43232e61-5925-4fcf-9ce6-a3092725958f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:56.246947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43232e61-5925-4fcf-9ce6-a3092725958f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:56.247217 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43232e61-5925-4fcf-9ce6-a3092725958f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:56.257017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43232e61-5925-4fcf-9ce6-a3092725958f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-43232e61-5925-4fcf-9ce6-a3092725958f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:05:56.265038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:56.266151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:05:56.267207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 596/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:05:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:56.272344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba5b0a-14ce-4507-88a6-63cb00d158b3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:56.372807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba5b0a-14ce-4507-88a6-63cb00d158b3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:56.400387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:56.406286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a888ea-6d4f-4616-8618-b05fa19d100a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:56.406943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a888ea-6d4f-4616-8618-b05fa19d100a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:56.407222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a888ea-6d4f-4616-8618-b05fa19d100a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:56.407491 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a888ea-6d4f-4616-8618-b05fa19d100a None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:56.408216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a888ea-6d4f-4616-8618-b05fa19d100a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:56.408518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a888ea-6d4f-4616-8618-b05fa19d100a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:56.409896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a888ea-6d4f-4616-8618-b05fa19d100a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:56.418147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 593/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:05:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:56.424207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43232e61-5925-4fcf-9ce6-a3092725958f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:56.456058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:56.523749 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43232e61-5925-4fcf-9ce6-a3092725958f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:56.557783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787db46-0217-4424-bd60-21e19e089ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:56.558637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787db46-0217-4424-bd60-21e19e089ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:56.558914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787db46-0217-4424-bd60-21e19e089ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:56.559180 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787db46-0217-4424-bd60-21e19e089ba7 None admin] PATH_INFO: `/v3/users/a87147222a43464cb334ee15e9d3dc07` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:56.560369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787db46-0217-4424-bd60-21e19e089ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=a87147222a43464cb334ee15e9d3dc07)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:56.561851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787db46-0217-4424-bd60-21e19e089ba7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:56.563457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 597/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:56 2022] GET /identity/v3/users/a87147222a43464cb334ee15e9d3dc07 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:56.569313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a888ea-6d4f-4616-8618-b05fa19d100a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:56.670599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a888ea-6d4f-4616-8618-b05fa19d100a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:56.703350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e928c-b033-4e3c-938f-8c833999d843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:56.704197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e928c-b033-4e3c-938f-8c833999d843 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:56.704476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e928c-b033-4e3c-938f-8c833999d843 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:56.704748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e928c-b033-4e3c-938f-8c833999d843 None admin] PATH_INFO: `/v3/projects/f9fefb0e2616494b9164de20d99a2422` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:56.705939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e928c-b033-4e3c-938f-8c833999d843 None admin] RBAC: Authorizing `identity:get_project(project_id=f9fefb0e2616494b9164de20d99a2422)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:56.707366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e928c-b033-4e3c-938f-8c833999d843 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:56.709055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 594/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:05:56 2022] GET /identity/v3/projects/f9fefb0e2616494b9164de20d99a2422 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:56.715983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787db46-0217-4424-bd60-21e19e089ba7 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:56.810743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:56.813686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787db46-0217-4424-bd60-21e19e089ba7 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:56.847591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb2851-3ad1-4695-aff5-2abb8b3e9170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:56.848463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb2851-3ad1-4695-aff5-2abb8b3e9170 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:56.848743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb2851-3ad1-4695-aff5-2abb8b3e9170 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:56.849011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb2851-3ad1-4695-aff5-2abb8b3e9170 None admin] PATH_INFO: `/v3/projects/f9fefb0e2616494b9164de20d99a2422/users/a87147222a43464cb334ee15e9d3dc07/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:56.850953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb2851-3ad1-4695-aff5-2abb8b3e9170 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9fefb0e2616494b9164de20d99a2422, user_id=a87147222a43464cb334ee15e9d3dc07, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:56.852398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb2851-3ad1-4695-aff5-2abb8b3e9170 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:56.865261 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 598/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:05:56 2022] PUT /identity/v3/projects/f9fefb0e2616494b9164de20d99a2422/users/a87147222a43464cb334ee15e9d3dc07/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:59.054024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bd684-a8a9-4f4c-a08e-3c3bafff97bd None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.054615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bd684-a8a9-4f4c-a08e-3c3bafff97bd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.054615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bd684-a8a9-4f4c-a08e-3c3bafff97bd None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.055948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 595/1193] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:59.061854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e29e3-be59-44bc-a4da-711cbee9d6b4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.062270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e29e3-be59-44bc-a4da-711cbee9d6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.062401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e29e3-be59-44bc-a4da-711cbee9d6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.078945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-002e29e3-be59-44bc-a4da-711cbee9d6b4 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:59.080129 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-002e29e3-be59-44bc-a4da-711cbee9d6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.144840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 599/1194] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:05:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:05:59.169380 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bd684-a8a9-4f4c-a08e-3c3bafff97bd None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.227859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:59.269582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bd684-a8a9-4f4c-a08e-3c3bafff97bd None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.303718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e15bb-753d-4615-bf40-3cc1ce08fa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.304548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e15bb-753d-4615-bf40-3cc1ce08fa54 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.304769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e15bb-753d-4615-bf40-3cc1ce08fa54 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.305032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e15bb-753d-4615-bf40-3cc1ce08fa54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.315026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e15bb-753d-4615-bf40-3cc1ce08fa54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-5e4e15bb-753d-4615-bf40-3cc1ce08fa54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:05:59.322806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:05:59.324061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:05:59.325023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 596/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:05:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:05:59.330417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e29e3-be59-44bc-a4da-711cbee9d6b4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.421170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:59.431511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e29e3-be59-44bc-a4da-711cbee9d6b4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.464464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634461-c4bb-4d3d-b411-9fa4eb5073cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.465125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634461-c4bb-4d3d-b411-9fa4eb5073cd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.465406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634461-c4bb-4d3d-b411-9fa4eb5073cd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.465680 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634461-c4bb-4d3d-b411-9fa4eb5073cd None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.466444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634461-c4bb-4d3d-b411-9fa4eb5073cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:59.466862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634461-c4bb-4d3d-b411-9fa4eb5073cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.468214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634461-c4bb-4d3d-b411-9fa4eb5073cd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.476067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 600/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:05:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:59.482730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e15bb-753d-4615-bf40-3cc1ce08fa54 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.588227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e15bb-753d-4615-bf40-3cc1ce08fa54 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.608907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:59.628141 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a72e4-2bac-4efe-b0fb-5e12cb657a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.629023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a72e4-2bac-4efe-b0fb-5e12cb657a5b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.629303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a72e4-2bac-4efe-b0fb-5e12cb657a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.629575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a72e4-2bac-4efe-b0fb-5e12cb657a5b None admin] PATH_INFO: `/v3/users/a87147222a43464cb334ee15e9d3dc07` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.631252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a72e4-2bac-4efe-b0fb-5e12cb657a5b None admin] RBAC: Authorizing `identity:get_user(user_id=a87147222a43464cb334ee15e9d3dc07)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.633483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a72e4-2bac-4efe-b0fb-5e12cb657a5b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.635879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 597/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 18:05:59 2022] GET /identity/v3/users/a87147222a43464cb334ee15e9d3dc07 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:59.642584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634461-c4bb-4d3d-b411-9fa4eb5073cd None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.743725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634461-c4bb-4d3d-b411-9fa4eb5073cd None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.776706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c006ff4-bdd6-45e6-a9ab-5ac710f2d4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.777334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c006ff4-bdd6-45e6-a9ab-5ac710f2d4bb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.777609 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c006ff4-bdd6-45e6-a9ab-5ac710f2d4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.778033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c006ff4-bdd6-45e6-a9ab-5ac710f2d4bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.799245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:59.844838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c006ff4-bdd6-45e6-a9ab-5ac710f2d4bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.846291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c006ff4-bdd6-45e6-a9ab-5ac710f2d4bb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.941917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 601/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:05:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:05:59.947795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a72e4-2bac-4efe-b0fb-5e12cb657a5b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:00.048375 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a72e4-2bac-4efe-b0fb-5e12cb657a5b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:00.081184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84390da1-0251-460f-8e07-71faa2ef1193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:00.082057 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84390da1-0251-460f-8e07-71faa2ef1193 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:00.082336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84390da1-0251-460f-8e07-71faa2ef1193 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:00.082606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84390da1-0251-460f-8e07-71faa2ef1193 None admin] PATH_INFO: `/v3/projects/f9fefb0e2616494b9164de20d99a2422` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:00.083774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84390da1-0251-460f-8e07-71faa2ef1193 None admin] RBAC: Authorizing `identity:get_project(project_id=f9fefb0e2616494b9164de20d99a2422)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:00.085183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84390da1-0251-460f-8e07-71faa2ef1193 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:00.086913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 598/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 18:05:59 2022] GET /identity/v3/projects/f9fefb0e2616494b9164de20d99a2422 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:06:00.093271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c006ff4-bdd6-45e6-a9ab-5ac710f2d4bb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:00.145082 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:00.191362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c006ff4-bdd6-45e6-a9ab-5ac710f2d4bb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:00.224543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021876be-eb78-45ba-8351-afaa9d21594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:00.225181 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021876be-eb78-45ba-8351-afaa9d21594b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:00.225464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021876be-eb78-45ba-8351-afaa9d21594b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:00.225759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021876be-eb78-45ba-8351-afaa9d21594b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:00.226579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021876be-eb78-45ba-8351-afaa9d21594b None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=f9fefb0e2616494b9164de20d99a2422, user.id=a87147222a43464cb334ee15e9d3dc07) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:00.226890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021876be-eb78-45ba-8351-afaa9d21594b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:00.228333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021876be-eb78-45ba-8351-afaa9d21594b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:00.234677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 602/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:06:00 2022] GET /identity/v3/role_assignments?user.id=a87147222a43464cb334ee15e9d3dc07&scope.project.id=f9fefb0e2616494b9164de20d99a2422&role.id=584beecb022b422f820e77f7623ada5e => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:06:10.077554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=72 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:10.232407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=42 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:03.119161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf2c01-8615-44d2-afd9-df0dcf45958d None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:03.120282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf2c01-8615-44d2-afd9-df0dcf45958d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:03.120282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf2c01-8615-44d2-afd9-df0dcf45958d None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:03.121837 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 599/1201] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:07:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:07:03.131083 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee6b70-d97e-4fb7-93b7-da332d2976b6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:03.132015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee6b70-d97e-4fb7-93b7-da332d2976b6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:03.132015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee6b70-d97e-4fb7-93b7-da332d2976b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:03.154850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8bee6b70-d97e-4fb7-93b7-da332d2976b6 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:03.156254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8bee6b70-d97e-4fb7-93b7-da332d2976b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:03.255654 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 603/1202] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:07:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:07:03.287126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf2c01-8615-44d2-afd9-df0dcf45958d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:03.462802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf2c01-8615-44d2-afd9-df0dcf45958d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:03.526694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e77d2f-0d30-4461-b827-7a031403d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:03.527622 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e77d2f-0d30-4461-b827-7a031403d417 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:03.527910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e77d2f-0d30-4461-b827-7a031403d417 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:03.528186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e77d2f-0d30-4461-b827-7a031403d417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:03.541953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e77d2f-0d30-4461-b827-7a031403d417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:03.543550 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e77d2f-0d30-4461-b827-7a031403d417 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-42e77d2f-0d30-4461-b827-7a031403d417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:03.567990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:03.569293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:03.570510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:03.570510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:03.570510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:03.570510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:03.574071 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 600/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:03.581149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee6b70-d97e-4fb7-93b7-da332d2976b6 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:03.717946 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee6b70-d97e-4fb7-93b7-da332d2976b6 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:03.752036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:03.760973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468e613-e101-43c5-b2c6-08ee8e466d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:03.762050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468e613-e101-43c5-b2c6-08ee8e466d2f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:03.762329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468e613-e101-43c5-b2c6-08ee8e466d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:03.762580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468e613-e101-43c5-b2c6-08ee8e466d2f None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:03.763313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468e613-e101-43c5-b2c6-08ee8e466d2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:03.763692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468e613-e101-43c5-b2c6-08ee8e466d2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:03.765027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468e613-e101-43c5-b2c6-08ee8e466d2f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:03.785011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 604/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:03.792352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e77d2f-0d30-4461-b827-7a031403d417 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:03.866828 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:03.960488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e77d2f-0d30-4461-b827-7a031403d417 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:04.013317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916c12c-465c-4f2c-909b-bd17717b4625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:04.014480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916c12c-465c-4f2c-909b-bd17717b4625 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:04.015101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916c12c-465c-4f2c-909b-bd17717b4625 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:04.015615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916c12c-465c-4f2c-909b-bd17717b4625 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:04.016458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916c12c-465c-4f2c-909b-bd17717b4625 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:04.018953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916c12c-465c-4f2c-909b-bd17717b4625 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:04.046891 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 601/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 18:07:03 2022] POST /identity/v3/users => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:07:06.662912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab165e-0dd4-49ee-99ca-06d904f65004 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:06.663971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab165e-0dd4-49ee-99ca-06d904f65004 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:06.663971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab165e-0dd4-49ee-99ca-06d904f65004 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:06.665230 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 605/1206] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:07:06.672364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d43bb-b800-40c5-b8a0-717d916e88a5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:06.672892 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d43bb-b800-40c5-b8a0-717d916e88a5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:06.673353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d43bb-b800-40c5-b8a0-717d916e88a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:06.696402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ca0d43bb-b800-40c5-b8a0-717d916e88a5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:06.698654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ca0d43bb-b800-40c5-b8a0-717d916e88a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:06.787771 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 602/1207] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:07:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:07:06.814516 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab165e-0dd4-49ee-99ca-06d904f65004 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:06.944318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab165e-0dd4-49ee-99ca-06d904f65004 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:06.989528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85066182-862a-4d49-b956-f93cabb3af1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:06.990679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85066182-862a-4d49-b956-f93cabb3af1e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:06.991158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85066182-862a-4d49-b956-f93cabb3af1e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:06.991622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85066182-862a-4d49-b956-f93cabb3af1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:07.011982 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85066182-862a-4d49-b956-f93cabb3af1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-85066182-862a-4d49-b956-f93cabb3af1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:07:07.027802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:07.035267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:07.036832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 606/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:07:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:07.038054 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d43bb-b800-40c5-b8a0-717d916e88a5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:07.090023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:07.188904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d43bb-b800-40c5-b8a0-717d916e88a5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:07.239425 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e00ac-98b2-4e17-92ce-651682cbc427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:07.240101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e00ac-98b2-4e17-92ce-651682cbc427 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:07.240434 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e00ac-98b2-4e17-92ce-651682cbc427 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:07.240771 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e00ac-98b2-4e17-92ce-651682cbc427 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:07.241629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e00ac-98b2-4e17-92ce-651682cbc427 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:07.242082 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e00ac-98b2-4e17-92ce-651682cbc427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:07.243642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e00ac-98b2-4e17-92ce-651682cbc427 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:07.254525 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 603/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:07:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:07.260605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85066182-862a-4d49-b956-f93cabb3af1e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:07.348614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:07.417184 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85066182-862a-4d49-b956-f93cabb3af1e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:07.464374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd042683-4b76-411b-94ef-bd81d170e148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:07.465163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd042683-4b76-411b-94ef-bd81d170e148 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:07.465437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd042683-4b76-411b-94ef-bd81d170e148 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:07.465705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd042683-4b76-411b-94ef-bd81d170e148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:07.477116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd042683-4b76-411b-94ef-bd81d170e148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:07.478642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd042683-4b76-411b-94ef-bd81d170e148 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-cd042683-4b76-411b-94ef-bd81d170e148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:07.498156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:07.499435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:07.500587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:07.500587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:07.500587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:07.500587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:07.500587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 607/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:07.505515 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e00ac-98b2-4e17-92ce-651682cbc427 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:07.651569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:07.655586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e00ac-98b2-4e17-92ce-651682cbc427 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:07.697636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787c3fc-2230-407a-bf12-e749df34d022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:07.700312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787c3fc-2230-407a-bf12-e749df34d022 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:07.700312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787c3fc-2230-407a-bf12-e749df34d022 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:07.700312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787c3fc-2230-407a-bf12-e749df34d022 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:07.700540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787c3fc-2230-407a-bf12-e749df34d022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:07.700917 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787c3fc-2230-407a-bf12-e749df34d022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:07.703523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787c3fc-2230-407a-bf12-e749df34d022 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:07.724878 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 604/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:07.726441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd042683-4b76-411b-94ef-bd81d170e148 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:07.868454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd042683-4b76-411b-94ef-bd81d170e148 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:07.868955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:07.923020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173930b-eb14-48a7-81be-72a532e38755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:07.923842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173930b-eb14-48a7-81be-72a532e38755 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:07.924128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173930b-eb14-48a7-81be-72a532e38755 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:07.924399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173930b-eb14-48a7-81be-72a532e38755 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:07.934857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173930b-eb14-48a7-81be-72a532e38755 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:07.936345 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173930b-eb14-48a7-81be-72a532e38755 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-5173930b-eb14-48a7-81be-72a532e38755 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:07:07.958671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:07.960094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:07.961060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:07.961060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:07.961060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:07:07.961060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:07:07.961060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:07:07.961060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:07:07.961060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:07:07.961060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:07:07.961060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 18:07:07.961060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:07.961060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 608/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:07.961878 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787c3fc-2230-407a-bf12-e749df34d022 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:08.107458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787c3fc-2230-407a-bf12-e749df34d022 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:08.147608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc5d2f-396e-4c36-9d6a-134f778e0007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:08.148339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc5d2f-396e-4c36-9d6a-134f778e0007 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:08.148620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc5d2f-396e-4c36-9d6a-134f778e0007 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:08.148888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc5d2f-396e-4c36-9d6a-134f778e0007 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:08.149744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc5d2f-396e-4c36-9d6a-134f778e0007 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:08.150061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc5d2f-396e-4c36-9d6a-134f778e0007 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:08.152263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc5d2f-396e-4c36-9d6a-134f778e0007 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:08.173522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 605/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Thu May 26 18:07:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:08.179343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173930b-eb14-48a7-81be-72a532e38755 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:08.348809 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173930b-eb14-48a7-81be-72a532e38755 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:08.393461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320cb3f-690c-464f-934c-3e0c33c4cb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:08.394178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320cb3f-690c-464f-934c-3e0c33c4cb6d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:08.394459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320cb3f-690c-464f-934c-3e0c33c4cb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:08.394729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320cb3f-690c-464f-934c-3e0c33c4cb6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:08.433500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:08.477627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320cb3f-690c-464f-934c-3e0c33c4cb6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:08.479548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320cb3f-690c-464f-934c-3e0c33c4cb6d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:08.601699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 609/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:07:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 18:07:08.607969 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc5d2f-396e-4c36-9d6a-134f778e0007 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:08.630034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:08.746832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc5d2f-396e-4c36-9d6a-134f778e0007 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:08.786254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fc553-b454-4ead-95bf-4411455f0a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:08.787404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fc553-b454-4ead-95bf-4411455f0a1a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:08.787762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fc553-b454-4ead-95bf-4411455f0a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:08.788014 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fc553-b454-4ead-95bf-4411455f0a1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:08.800576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fc553-b454-4ead-95bf-4411455f0a1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:08.802089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fc553-b454-4ead-95bf-4411455f0a1a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-fb5fc553-b454-4ead-95bf-4411455f0a1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:08.821711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:08.824657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:08.825586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:08.825586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:08.825586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:08.825586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:08.825586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 606/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:07:08.829824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320cb3f-690c-464f-934c-3e0c33c4cb6d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:08.912035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:08.967041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320cb3f-690c-464f-934c-3e0c33c4cb6d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:09.019198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb5988-6192-4784-8ee3-1634fa98cc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:09.019198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb5988-6192-4784-8ee3-1634fa98cc02 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:09.019693 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb5988-6192-4784-8ee3-1634fa98cc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:09.019956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb5988-6192-4784-8ee3-1634fa98cc02 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:09.020816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb5988-6192-4784-8ee3-1634fa98cc02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:09.021448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb5988-6192-4784-8ee3-1634fa98cc02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:09.022879 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb5988-6192-4784-8ee3-1634fa98cc02 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:09.044705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 610/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:09.050776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fc553-b454-4ead-95bf-4411455f0a1a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:09.154178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:09.185493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fc553-b454-4ead-95bf-4411455f0a1a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:09.226465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2d05d-2f08-41e9-8439-82b22c5c5643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:09.227303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2d05d-2f08-41e9-8439-82b22c5c5643 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:09.227580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2d05d-2f08-41e9-8439-82b22c5c5643 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:09.227849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2d05d-2f08-41e9-8439-82b22c5c5643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:09.235787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee2d05d-2f08-41e9-8439-82b22c5c5643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:09.237257 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee2d05d-2f08-41e9-8439-82b22c5c5643 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-bee2d05d-2f08-41e9-8439-82b22c5c5643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:07:09.254886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:09.256374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:09.257573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:07:09.257573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:07:09.257573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:07:09.257573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:07:09.257573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:07:09.257573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:07:09.257573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:07:09.257573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:09.262741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb5988-6192-4784-8ee3-1634fa98cc02 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:09.263211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 607/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:07:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:09.395006 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb5988-6192-4784-8ee3-1634fa98cc02 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:09.431661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:09.454653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbac0b-1903-40dc-8663-1e37f5b6428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:09.455322 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbac0b-1903-40dc-8663-1e37f5b6428d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:09.456506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbac0b-1903-40dc-8663-1e37f5b6428d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:09.456506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbac0b-1903-40dc-8663-1e37f5b6428d None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:09.456845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbac0b-1903-40dc-8663-1e37f5b6428d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:09.457003 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbac0b-1903-40dc-8663-1e37f5b6428d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:09.458375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbac0b-1903-40dc-8663-1e37f5b6428d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:09.479500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 611/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:07:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:09.487405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2d05d-2f08-41e9-8439-82b22c5c5643 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:09.622034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2d05d-2f08-41e9-8439-82b22c5c5643 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:09.665238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:09.675030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e7370-6299-40fb-b64f-a82e3fe3d2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:09.675953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e7370-6299-40fb-b64f-a82e3fe3d2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:09.676573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e7370-6299-40fb-b64f-a82e3fe3d2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:09.676868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e7370-6299-40fb-b64f-a82e3fe3d2e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:09.677654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e7370-6299-40fb-b64f-a82e3fe3d2e7 None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=7ee40bcb7905490c9ad3e39e7a93b122) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:09.677996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e7370-6299-40fb-b64f-a82e3fe3d2e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:09.679442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e7370-6299-40fb-b64f-a82e3fe3d2e7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:09.686710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 608/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:07:09 2022] GET /identity/v3/role_assignments?user.id=7ee40bcb7905490c9ad3e39e7a93b122&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:12.085439 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf6262-1d76-4d16-929c-579a119e2045 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:12.086323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf6262-1d76-4d16-929c-579a119e2045 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:12.086323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf6262-1d76-4d16-929c-579a119e2045 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:12.088798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 612/1220] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:07:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:07:12.095980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606f2e5-5959-4b55-93a2-ba335d236033 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:12.096704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606f2e5-5959-4b55-93a2-ba335d236033 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:12.096704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606f2e5-5959-4b55-93a2-ba335d236033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:12.115861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d606f2e5-5959-4b55-93a2-ba335d236033 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:12.117078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d606f2e5-5959-4b55-93a2-ba335d236033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:12.202863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 609/1221] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:07:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 18:07:12.227057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf6262-1d76-4d16-929c-579a119e2045 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:12.353716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf6262-1d76-4d16-929c-579a119e2045 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:12.407438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b09553-819b-4111-959f-dfc4c95d2379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:12.408236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b09553-819b-4111-959f-dfc4c95d2379 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:12.408517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b09553-819b-4111-959f-dfc4c95d2379 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:12.408793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b09553-819b-4111-959f-dfc4c95d2379 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:12.421345 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b09553-819b-4111-959f-dfc4c95d2379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-31b09553-819b-4111-959f-dfc4c95d2379 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:07:12.431376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:12.438049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:12.439248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 613/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:07:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:12.441170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606f2e5-5959-4b55-93a2-ba335d236033 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:12.578312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606f2e5-5959-4b55-93a2-ba335d236033 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:12.610881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:12.624298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb6ac9-4cad-4d7d-8070-0c8285e3aa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:12.625344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb6ac9-4cad-4d7d-8070-0c8285e3aa67 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:12.625839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb6ac9-4cad-4d7d-8070-0c8285e3aa67 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:12.626291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb6ac9-4cad-4d7d-8070-0c8285e3aa67 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:12.627497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb6ac9-4cad-4d7d-8070-0c8285e3aa67 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:12.627911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb6ac9-4cad-4d7d-8070-0c8285e3aa67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:12.629888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb6ac9-4cad-4d7d-8070-0c8285e3aa67 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:12.640065 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 610/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:07:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:12.647363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b09553-819b-4111-959f-dfc4c95d2379 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:12.715265 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:12.783806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b09553-819b-4111-959f-dfc4c95d2379 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:12.832956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d2972-3f4e-4ca0-8794-963b9048b1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:12.833931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d2972-3f4e-4ca0-8794-963b9048b1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:12.834285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d2972-3f4e-4ca0-8794-963b9048b1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:12.834566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d2972-3f4e-4ca0-8794-963b9048b1a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:12.847336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d2972-3f4e-4ca0-8794-963b9048b1a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:12.849127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d2972-3f4e-4ca0-8794-963b9048b1a8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-4b2d2972-3f4e-4ca0-8794-963b9048b1a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:12.868543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:12.869984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:12.870891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:12.870891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:12.870891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:12.870891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:12.870891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 614/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:12.876601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb6ac9-4cad-4d7d-8070-0c8285e3aa67 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:13.009630 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb6ac9-4cad-4d7d-8070-0c8285e3aa67 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:13.051834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd83c5-260a-4629-87df-cf7f3195c004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:13.052577 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd83c5-260a-4629-87df-cf7f3195c004 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:13.052791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd83c5-260a-4629-87df-cf7f3195c004 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:13.053186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd83c5-260a-4629-87df-cf7f3195c004 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:13.054093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd83c5-260a-4629-87df-cf7f3195c004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:13.054450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd83c5-260a-4629-87df-cf7f3195c004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:13.056058 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd83c5-260a-4629-87df-cf7f3195c004 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:13.069909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 611/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:13.076951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d2972-3f4e-4ca0-8794-963b9048b1a8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:13.194682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:13.204366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d2972-3f4e-4ca0-8794-963b9048b1a8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:13.245802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb4af1-38e4-41c8-833e-1d6749c55ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:13.246546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb4af1-38e4-41c8-833e-1d6749c55ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:13.246829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb4af1-38e4-41c8-833e-1d6749c55ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:13.247167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb4af1-38e4-41c8-833e-1d6749c55ae8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:13.258115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb4af1-38e4-41c8-833e-1d6749c55ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:13.259661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb4af1-38e4-41c8-833e-1d6749c55ae8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-b9bb4af1-38e4-41c8-833e-1d6749c55ae8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:07:13.275652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:13.282473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:13.283517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:13.283517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:13.283517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:07:13.283517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:07:13.283517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:07:13.283517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:07:13.283517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:07:13.283517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:07:13.283517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 18:07:13.283517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:13.283517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 615/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:13.284129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd83c5-260a-4629-87df-cf7f3195c004 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:13.335916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:13.402624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd83c5-260a-4629-87df-cf7f3195c004 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:13.440858 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257ff07-1e7d-4312-85c3-2d34730d28cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:13.441773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257ff07-1e7d-4312-85c3-2d34730d28cd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:13.442062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257ff07-1e7d-4312-85c3-2d34730d28cd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:13.442260 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257ff07-1e7d-4312-85c3-2d34730d28cd None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:13.443032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257ff07-1e7d-4312-85c3-2d34730d28cd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:13.443347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257ff07-1e7d-4312-85c3-2d34730d28cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:13.444876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257ff07-1e7d-4312-85c3-2d34730d28cd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:13.475614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 612/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Thu May 26 18:07:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:13.476824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb4af1-38e4-41c8-833e-1d6749c55ae8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:13.610513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb4af1-38e4-41c8-833e-1d6749c55ae8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:13.645460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ea1d5-c859-402b-9102-d02de4a3463e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:13.646364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ea1d5-c859-402b-9102-d02de4a3463e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:13.646633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ea1d5-c859-402b-9102-d02de4a3463e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:13.646945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ea1d5-c859-402b-9102-d02de4a3463e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:13.659071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ea1d5-c859-402b-9102-d02de4a3463e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:13.660578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ea1d5-c859-402b-9102-d02de4a3463e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a22ea1d5-c859-402b-9102-d02de4a3463e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:13.681549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:13.683407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:13.684327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:13.684327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:13.684327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:13.684327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:13.684327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 616/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 18:07:13.689320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257ff07-1e7d-4312-85c3-2d34730d28cd None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:13.810767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257ff07-1e7d-4312-85c3-2d34730d28cd None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:13.813643 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:13.859650 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e477311-e012-4175-b6aa-6fafaad27001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:13.860329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e477311-e012-4175-b6aa-6fafaad27001 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:13.860616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e477311-e012-4175-b6aa-6fafaad27001 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:13.860888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e477311-e012-4175-b6aa-6fafaad27001 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:13.861620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e477311-e012-4175-b6aa-6fafaad27001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:13.861986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e477311-e012-4175-b6aa-6fafaad27001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:13.863348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e477311-e012-4175-b6aa-6fafaad27001 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:13.875736 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 613/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:07:13.881780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ea1d5-c859-402b-9102-d02de4a3463e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:13.896214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:14.027006 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ea1d5-c859-402b-9102-d02de4a3463e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:14.068021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51099532-df98-4fde-af34-c8ecb59c2517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:14.068735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51099532-df98-4fde-af34-c8ecb59c2517 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.069014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51099532-df98-4fde-af34-c8ecb59c2517 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.069287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51099532-df98-4fde-af34-c8ecb59c2517 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.079968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51099532-df98-4fde-af34-c8ecb59c2517 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:14.082605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51099532-df98-4fde-af34-c8ecb59c2517 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-51099532-df98-4fde-af34-c8ecb59c2517 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:07:14.100325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:14.101375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:14.102326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:07:14.102326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:07:14.102326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:07:14.102326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:07:14.102326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:07:14.102326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:07:14.102326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:07:14.102326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:14.102326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 617/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:07:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:14.113258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e477311-e012-4175-b6aa-6fafaad27001 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:14.241348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e477311-e012-4175-b6aa-6fafaad27001 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:14.287971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19529b22-ea40-458a-a138-96176e4bbf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:14.288747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19529b22-ea40-458a-a138-96176e4bbf95 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.289285 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19529b22-ea40-458a-a138-96176e4bbf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.289561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19529b22-ea40-458a-a138-96176e4bbf95 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.290438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19529b22-ea40-458a-a138-96176e4bbf95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:14.290755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19529b22-ea40-458a-a138-96176e4bbf95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:14.292637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19529b22-ea40-458a-a138-96176e4bbf95 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:14.310371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 614/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:07:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:14.320023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51099532-df98-4fde-af34-c8ecb59c2517 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:14.428124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:14.480847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51099532-df98-4fde-af34-c8ecb59c2517 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:14.522730 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c93b2-05fd-4fbd-b3ff-4e562d1af15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:14.523678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c93b2-05fd-4fbd-b3ff-4e562d1af15f None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.523960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c93b2-05fd-4fbd-b3ff-4e562d1af15f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.524236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c93b2-05fd-4fbd-b3ff-4e562d1af15f None admin] PATH_INFO: `/v3/projects/4be4045293754773bb52bbb82546a1db/users/7ee40bcb7905490c9ad3e39e7a93b122/roles/d9bae602bae74c71b6c858434a426a3d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.545282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c93b2-05fd-4fbd-b3ff-4e562d1af15f None admin] RBAC: Authorizing `identity:create_grant(project_id=4be4045293754773bb52bbb82546a1db, user_id=7ee40bcb7905490c9ad3e39e7a93b122, role_id=d9bae602bae74c71b6c858434a426a3d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:14.548397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c93b2-05fd-4fbd-b3ff-4e562d1af15f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:14.564358 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 618/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:07:14 2022] PUT /identity/v3/projects/4be4045293754773bb52bbb82546a1db/users/7ee40bcb7905490c9ad3e39e7a93b122/roles/d9bae602bae74c71b6c858434a426a3d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:07:17.063294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52036780-8d46-4981-a480-9ba060642e64 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:17.063294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52036780-8d46-4981-a480-9ba060642e64 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:17.063294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52036780-8d46-4981-a480-9ba060642e64 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:17.063294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 615/1233] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:07:17.072347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44aa975-d63a-4a3d-8aff-d6bd7ff15f70 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:17.073045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44aa975-d63a-4a3d-8aff-d6bd7ff15f70 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:17.073045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44aa975-d63a-4a3d-8aff-d6bd7ff15f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:17.091094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e44aa975-d63a-4a3d-8aff-d6bd7ff15f70 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:17.092321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e44aa975-d63a-4a3d-8aff-d6bd7ff15f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:17.178103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 619/1234] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:07:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:07:17.214910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52036780-8d46-4981-a480-9ba060642e64 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:17.251882 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:17.357112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52036780-8d46-4981-a480-9ba060642e64 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:17.401285 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4540199-f62c-408f-b098-19728c5599d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:17.402904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4540199-f62c-408f-b098-19728c5599d9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:17.403291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4540199-f62c-408f-b098-19728c5599d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:17.403666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4540199-f62c-408f-b098-19728c5599d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:17.422132 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4540199-f62c-408f-b098-19728c5599d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e4540199-f62c-408f-b098-19728c5599d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:07:17.436201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:17.437247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:17.438149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 616/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 18:07:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:17.444117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44aa975-d63a-4a3d-8aff-d6bd7ff15f70 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:17.500685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:17.578962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44aa975-d63a-4a3d-8aff-d6bd7ff15f70 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:17.622844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ab98c-39eb-40b5-bcac-2e1cf1fbfd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:17.623554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ab98c-39eb-40b5-bcac-2e1cf1fbfd23 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:17.623832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ab98c-39eb-40b5-bcac-2e1cf1fbfd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:17.624099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ab98c-39eb-40b5-bcac-2e1cf1fbfd23 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:17.624892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ab98c-39eb-40b5-bcac-2e1cf1fbfd23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:17.625201 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ab98c-39eb-40b5-bcac-2e1cf1fbfd23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:17.626666 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ab98c-39eb-40b5-bcac-2e1cf1fbfd23 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:17.637528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 620/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 18:07:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:17.643701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4540199-f62c-408f-b098-19728c5599d9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:17.747672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:17.770332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4540199-f62c-408f-b098-19728c5599d9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:17.815065 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023dcb0-606a-4baf-93ac-13da3641f323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:17.816077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023dcb0-606a-4baf-93ac-13da3641f323 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:17.816458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023dcb0-606a-4baf-93ac-13da3641f323 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:17.816768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023dcb0-606a-4baf-93ac-13da3641f323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:17.831363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023dcb0-606a-4baf-93ac-13da3641f323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:17.831547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023dcb0-606a-4baf-93ac-13da3641f323 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-9023dcb0-606a-4baf-93ac-13da3641f323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:17.851841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:17.852995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:17.854026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:17.854026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:17.854026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:17.854026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:17.854026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 617/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:17.859974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ab98c-39eb-40b5-bcac-2e1cf1fbfd23 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:17.980103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ab98c-39eb-40b5-bcac-2e1cf1fbfd23 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:17.989498 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:18.033839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c3616-543a-4040-b87a-592c26aa7598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:18.033839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c3616-543a-4040-b87a-592c26aa7598 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:18.033839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c3616-543a-4040-b87a-592c26aa7598 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:18.033839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c3616-543a-4040-b87a-592c26aa7598 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:18.034906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c3616-543a-4040-b87a-592c26aa7598 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:18.034906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c3616-543a-4040-b87a-592c26aa7598 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:18.034906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c3616-543a-4040-b87a-592c26aa7598 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:18.047338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 621/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:07:18.053696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023dcb0-606a-4baf-93ac-13da3641f323 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:18.179798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023dcb0-606a-4baf-93ac-13da3641f323 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:18.221667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683393ee-df25-4722-8e3d-f274b64d4447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:18.222525 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683393ee-df25-4722-8e3d-f274b64d4447 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:18.222802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683393ee-df25-4722-8e3d-f274b64d4447 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:18.223073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683393ee-df25-4722-8e3d-f274b64d4447 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:18.226235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:18.233364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683393ee-df25-4722-8e3d-f274b64d4447 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:18.235430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683393ee-df25-4722-8e3d-f274b64d4447 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-683393ee-df25-4722-8e3d-f274b64d4447 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:07:18.252802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:18.254730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:18.255695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:18.255695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:18.255695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:07:18.255695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:07:18.255695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:07:18.255695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:07:18.255695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:07:18.255695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:07:18.255695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 18:07:18.255695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:18.255695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 618/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:18.261105 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c3616-543a-4040-b87a-592c26aa7598 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:18.417236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c3616-543a-4040-b87a-592c26aa7598 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:18.463655 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab44aa-8321-4a9c-b858-1a83ca224b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:18.464444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab44aa-8321-4a9c-b858-1a83ca224b56 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:18.464871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab44aa-8321-4a9c-b858-1a83ca224b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:18.465209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab44aa-8321-4a9c-b858-1a83ca224b56 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:18.466263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab44aa-8321-4a9c-b858-1a83ca224b56 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:18.466648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab44aa-8321-4a9c-b858-1a83ca224b56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:18.468486 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab44aa-8321-4a9c-b858-1a83ca224b56 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:18.490838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 622/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Thu May 26 18:07:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:07:18.498253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683393ee-df25-4722-8e3d-f274b64d4447 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:18.656695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683393ee-df25-4722-8e3d-f274b64d4447 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:18.711804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbf7c8-654e-4261-acbc-bcf0ecec4903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:18.712715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbf7c8-654e-4261-acbc-bcf0ecec4903 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:18.713858 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbf7c8-654e-4261-acbc-bcf0ecec4903 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:18.714479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbf7c8-654e-4261-acbc-bcf0ecec4903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:18.809799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:18.826897 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbf7c8-654e-4261-acbc-bcf0ecec4903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:18.830029 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbf7c8-654e-4261-acbc-bcf0ecec4903 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:18.985010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 619/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:07:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:07:18.992668 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab44aa-8321-4a9c-b858-1a83ca224b56 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:19.037840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:19.141034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab44aa-8321-4a9c-b858-1a83ca224b56 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:19.180522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7ff8e-c1d0-417e-abe9-d774f9007058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:19.181313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7ff8e-c1d0-417e-abe9-d774f9007058 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:19.181590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7ff8e-c1d0-417e-abe9-d774f9007058 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:19.182456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7ff8e-c1d0-417e-abe9-d774f9007058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:19.194887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7ff8e-c1d0-417e-abe9-d774f9007058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:19.196765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7ff8e-c1d0-417e-abe9-d774f9007058 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-aec7ff8e-c1d0-417e-abe9-d774f9007058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:19.218835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:19.220059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:19.221257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:19.221257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:19.221257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:19.221257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:19.223092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 623/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:19.229097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbf7c8-654e-4261-acbc-bcf0ecec4903 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:19.352435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:19.361142 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbf7c8-654e-4261-acbc-bcf0ecec4903 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:19.411750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4406da-2e48-43e5-8f0d-f3613289a266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:19.412650 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4406da-2e48-43e5-8f0d-f3613289a266 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:19.412933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4406da-2e48-43e5-8f0d-f3613289a266 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:19.425901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4406da-2e48-43e5-8f0d-f3613289a266 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:19.425901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4406da-2e48-43e5-8f0d-f3613289a266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:19.425901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4406da-2e48-43e5-8f0d-f3613289a266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:19.425901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4406da-2e48-43e5-8f0d-f3613289a266 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:19.439436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 620/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:07:19.446446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7ff8e-c1d0-417e-abe9-d774f9007058 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:19.552501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:19.581820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7ff8e-c1d0-417e-abe9-d774f9007058 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:19.636910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490d431-42b0-438e-9802-82b6aa4767bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:19.637759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490d431-42b0-438e-9802-82b6aa4767bd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:19.638046 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490d431-42b0-438e-9802-82b6aa4767bd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:19.638311 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490d431-42b0-438e-9802-82b6aa4767bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:19.646551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490d431-42b0-438e-9802-82b6aa4767bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:19.648346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490d431-42b0-438e-9802-82b6aa4767bd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-e490d431-42b0-438e-9802-82b6aa4767bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:07:19.668740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:19.670820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:19.672478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:07:19.672478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:07:19.672478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:07:19.672478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:07:19.672478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:07:19.672478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:07:19.672478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:07:19.672478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:19.672478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 624/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:07:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:19.676730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4406da-2e48-43e5-8f0d-f3613289a266 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:19.829383 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4406da-2e48-43e5-8f0d-f3613289a266 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:19.866510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a967b-9f91-4ff6-9afa-6601734f0848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:19.867197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a967b-9f91-4ff6-9afa-6601734f0848 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:19.867508 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a967b-9f91-4ff6-9afa-6601734f0848 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:19.867782 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a967b-9f91-4ff6-9afa-6601734f0848 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:19.868607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a967b-9f91-4ff6-9afa-6601734f0848 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:19.868911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a967b-9f91-4ff6-9afa-6601734f0848 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:19.870495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a967b-9f91-4ff6-9afa-6601734f0848 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:19.886110 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 621/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:07:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:19.892773 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490d431-42b0-438e-9802-82b6aa4767bd None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:20.031012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490d431-42b0-438e-9802-82b6aa4767bd None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:20.076389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd6d69-0fc9-49b7-a01d-3e247a5a6015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:20.077069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd6d69-0fc9-49b7-a01d-3e247a5a6015 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:20.077334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd6d69-0fc9-49b7-a01d-3e247a5a6015 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:20.077646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd6d69-0fc9-49b7-a01d-3e247a5a6015 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:20.078577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd6d69-0fc9-49b7-a01d-3e247a5a6015 None admin] RBAC: Adding query filter params (role.id=d9bae602bae74c71b6c858434a426a3d, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=7ee40bcb7905490c9ad3e39e7a93b122) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:20.078893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd6d69-0fc9-49b7-a01d-3e247a5a6015 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:20.080551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd6d69-0fc9-49b7-a01d-3e247a5a6015 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:20.087854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 625/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:07:19 2022] GET /identity/v3/role_assignments?user.id=7ee40bcb7905490c9ad3e39e7a93b122&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=d9bae602bae74c71b6c858434a426a3d => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:22.478412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd6777-f521-44e2-a6c8-f8b6d8d651fa None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:22.479010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd6777-f521-44e2-a6c8-f8b6d8d651fa None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:22.479010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd6777-f521-44e2-a6c8-f8b6d8d651fa None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:22.480606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 622/1247] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:07:22.487071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30dcff-d8ba-47fc-8290-1c779e091390 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:22.487071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30dcff-d8ba-47fc-8290-1c779e091390 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:22.487071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30dcff-d8ba-47fc-8290-1c779e091390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:22.493860 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:22.508539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8c30dcff-d8ba-47fc-8290-1c779e091390 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:22.510534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8c30dcff-d8ba-47fc-8290-1c779e091390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:22.616369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 626/1248] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:07:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:07:22.643717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd6777-f521-44e2-a6c8-f8b6d8d651fa None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:22.665492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:22.774830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd6777-f521-44e2-a6c8-f8b6d8d651fa None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:22.815694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ffb0a-fa42-4e20-9854-db6a262a3b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:22.816482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ffb0a-fa42-4e20-9854-db6a262a3b4e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:22.816867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ffb0a-fa42-4e20-9854-db6a262a3b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:22.817018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ffb0a-fa42-4e20-9854-db6a262a3b4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:22.830344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ffb0a-fa42-4e20-9854-db6a262a3b4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-bb0ffb0a-fa42-4e20-9854-db6a262a3b4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:07:22.840042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:22.841909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:22.843422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 623/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:07:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:22.850694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30dcff-d8ba-47fc-8290-1c779e091390 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:22.969141 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30dcff-d8ba-47fc-8290-1c779e091390 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:23.009646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:23.033673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231b215-fd4b-49fc-bb87-37911c3e2794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:23.035165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231b215-fd4b-49fc-bb87-37911c3e2794 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:23.035790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231b215-fd4b-49fc-bb87-37911c3e2794 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:23.036353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231b215-fd4b-49fc-bb87-37911c3e2794 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:23.038532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231b215-fd4b-49fc-bb87-37911c3e2794 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:23.039410 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231b215-fd4b-49fc-bb87-37911c3e2794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:23.044559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231b215-fd4b-49fc-bb87-37911c3e2794 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:23.058001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 627/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:07:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:23.065819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ffb0a-fa42-4e20-9854-db6a262a3b4e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:23.159937 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:23.185967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ffb0a-fa42-4e20-9854-db6a262a3b4e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:23.228250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94988300-b65f-4a7a-9831-61838d120e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:23.229707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94988300-b65f-4a7a-9831-61838d120e2e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:23.230214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94988300-b65f-4a7a-9831-61838d120e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:23.230563 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94988300-b65f-4a7a-9831-61838d120e2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:23.247223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94988300-b65f-4a7a-9831-61838d120e2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:23.248990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94988300-b65f-4a7a-9831-61838d120e2e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-94988300-b65f-4a7a-9831-61838d120e2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:23.271877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:23.272839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:23.273753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:23.273753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:23.273753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:23.273753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:23.273753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 624/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:23.278203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231b215-fd4b-49fc-bb87-37911c3e2794 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:23.410441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231b215-fd4b-49fc-bb87-37911c3e2794 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:23.457027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0aa2f2-0862-497a-a94d-9d5160e337ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:23.457703 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aa2f2-0862-497a-a94d-9d5160e337ec None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:23.458012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aa2f2-0862-497a-a94d-9d5160e337ec None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:23.458291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aa2f2-0862-497a-a94d-9d5160e337ec None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:23.459021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0aa2f2-0862-497a-a94d-9d5160e337ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:23.459347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0aa2f2-0862-497a-a94d-9d5160e337ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:23.460734 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0aa2f2-0862-497a-a94d-9d5160e337ec None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:23.477343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 628/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:23.485025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94988300-b65f-4a7a-9831-61838d120e2e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:23.611737 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94988300-b65f-4a7a-9831-61838d120e2e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:23.644440 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:23.653959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf15f0-cf2c-40bc-9263-5b84f72a7bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:23.655181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf15f0-cf2c-40bc-9263-5b84f72a7bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:23.655494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf15f0-cf2c-40bc-9263-5b84f72a7bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:23.655806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf15f0-cf2c-40bc-9263-5b84f72a7bd4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:23.665914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf15f0-cf2c-40bc-9263-5b84f72a7bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:23.668011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf15f0-cf2c-40bc-9263-5b84f72a7bd4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-baaf15f0-cf2c-40bc-9263-5b84f72a7bd4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:07:23.685690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:23.686896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:23.687876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:23.687876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:23.687876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:07:23.687876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:07:23.687876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:07:23.687876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:07:23.687876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:07:23.687876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:07:23.687876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 18:07:23.687876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:23.687876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 625/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:23.694240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0aa2f2-0862-497a-a94d-9d5160e337ec None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:23.767820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:23.834820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0aa2f2-0862-497a-a94d-9d5160e337ec None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:23.877370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd25493-02a0-40d6-b9cd-6cba5cd605a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:23.878081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd25493-02a0-40d6-b9cd-6cba5cd605a5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:23.878365 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd25493-02a0-40d6-b9cd-6cba5cd605a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:23.878640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd25493-02a0-40d6-b9cd-6cba5cd605a5 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:23.879798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd25493-02a0-40d6-b9cd-6cba5cd605a5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:23.880086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd25493-02a0-40d6-b9cd-6cba5cd605a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:23.881713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd25493-02a0-40d6-b9cd-6cba5cd605a5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:23.904402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 629/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Thu May 26 18:07:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:23.910224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf15f0-cf2c-40bc-9263-5b84f72a7bd4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:24.038139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf15f0-cf2c-40bc-9263-5b84f72a7bd4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:24.081581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd7820-6e4e-484d-9164-d60735a4095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:24.082335 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd7820-6e4e-484d-9164-d60735a4095b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:24.082860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd7820-6e4e-484d-9164-d60735a4095b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:24.083128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd7820-6e4e-484d-9164-d60735a4095b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:24.159176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:24.175714 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd7820-6e4e-484d-9164-d60735a4095b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:24.177455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd7820-6e4e-484d-9164-d60735a4095b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:24.304254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 626/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:07:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:07:24.310975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd25493-02a0-40d6-b9cd-6cba5cd605a5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:24.458659 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd25493-02a0-40d6-b9cd-6cba5cd605a5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:24.476920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:24.505753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525bd10-87e9-4196-b5f4-53f8c13ffa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:24.506583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525bd10-87e9-4196-b5f4-53f8c13ffa5b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:24.506870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525bd10-87e9-4196-b5f4-53f8c13ffa5b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:24.507143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525bd10-87e9-4196-b5f4-53f8c13ffa5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:24.520794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525bd10-87e9-4196-b5f4-53f8c13ffa5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:24.523023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525bd10-87e9-4196-b5f4-53f8c13ffa5b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-e525bd10-87e9-4196-b5f4-53f8c13ffa5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:24.546831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:24.548296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:24.549198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:24.549198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:24.549198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:24.549198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:24.549198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 630/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:24.555217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd7820-6e4e-484d-9164-d60735a4095b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:24.671848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:24.689174 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd7820-6e4e-484d-9164-d60735a4095b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:24.734367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be75ae-c37d-4acf-b2c1-00d404a3eff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:24.735103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be75ae-c37d-4acf-b2c1-00d404a3eff8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:24.735486 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be75ae-c37d-4acf-b2c1-00d404a3eff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:24.735806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be75ae-c37d-4acf-b2c1-00d404a3eff8 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:24.736732 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be75ae-c37d-4acf-b2c1-00d404a3eff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:24.737023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be75ae-c37d-4acf-b2c1-00d404a3eff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:24.738687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be75ae-c37d-4acf-b2c1-00d404a3eff8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:24.753777 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 627/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:24.759492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525bd10-87e9-4196-b5f4-53f8c13ffa5b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:24.894278 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525bd10-87e9-4196-b5f4-53f8c13ffa5b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:24.940440 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05430d-d329-4fe7-869f-be0212ee9ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:24.941624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05430d-d329-4fe7-869f-be0212ee9ace None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:24.942065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05430d-d329-4fe7-869f-be0212ee9ace None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:24.942585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05430d-d329-4fe7-869f-be0212ee9ace None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:24.953036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05430d-d329-4fe7-869f-be0212ee9ace None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:24.954864 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05430d-d329-4fe7-869f-be0212ee9ace None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9f05430d-d329-4fe7-869f-be0212ee9ace None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:07:24.970316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:24.971602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:24.974784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:07:24.974784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:07:24.974784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:07:24.974784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:07:24.974784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:07:24.974784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:07:24.974784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:07:24.974784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:24.974784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 631/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:07:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:24.977622 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be75ae-c37d-4acf-b2c1-00d404a3eff8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:25.112830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be75ae-c37d-4acf-b2c1-00d404a3eff8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:25.160931 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c506937-3a2f-4a23-bd21-f0c521037c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:25.161547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c506937-3a2f-4a23-bd21-f0c521037c0f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:25.161857 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c506937-3a2f-4a23-bd21-f0c521037c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:25.162244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c506937-3a2f-4a23-bd21-f0c521037c0f None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:25.163041 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c506937-3a2f-4a23-bd21-f0c521037c0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:25.163349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c506937-3a2f-4a23-bd21-f0c521037c0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:25.165039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c506937-3a2f-4a23-bd21-f0c521037c0f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:25.180914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 628/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:07:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:25.187544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05430d-d329-4fe7-869f-be0212ee9ace None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:25.217428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:25.328838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05430d-d329-4fe7-869f-be0212ee9ace None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:25.372602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff6ee1-75f3-4325-bcd7-5c5b28f6567c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:25.373267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff6ee1-75f3-4325-bcd7-5c5b28f6567c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:25.373550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff6ee1-75f3-4325-bcd7-5c5b28f6567c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:25.374413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff6ee1-75f3-4325-bcd7-5c5b28f6567c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:25.375246 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff6ee1-75f3-4325-bcd7-5c5b28f6567c None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=7ee40bcb7905490c9ad3e39e7a93b122) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:25.375586 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff6ee1-75f3-4325-bcd7-5c5b28f6567c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:25.377259 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff6ee1-75f3-4325-bcd7-5c5b28f6567c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:25.395578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 632/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:07:25 2022] GET /identity/v3/role_assignments?user.id=7ee40bcb7905490c9ad3e39e7a93b122&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=584beecb022b422f820e77f7623ada5e => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:27.788689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad5f70-1bb1-45b6-85a5-a07ceb8bcf74 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:27.789511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad5f70-1bb1-45b6-85a5-a07ceb8bcf74 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:27.789511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad5f70-1bb1-45b6-85a5-a07ceb8bcf74 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:27.790891 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 629/1261] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:07:27.797225 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916c7fc-ae34-4f54-b6cc-e5cc07d08083 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:27.797685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916c7fc-ae34-4f54-b6cc-e5cc07d08083 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:27.797798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916c7fc-ae34-4f54-b6cc-e5cc07d08083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:27.815526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5916c7fc-ae34-4f54-b6cc-e5cc07d08083 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:27.816807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5916c7fc-ae34-4f54-b6cc-e5cc07d08083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:27.886263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 633/1262] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:07:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:07:27.909099 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad5f70-1bb1-45b6-85a5-a07ceb8bcf74 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:27.921340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:28.017940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad5f70-1bb1-45b6-85a5-a07ceb8bcf74 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:28.054660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a56b80-ed77-45c5-8900-50616d8a0d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:28.055365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a56b80-ed77-45c5-8900-50616d8a0d20 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:28.055646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a56b80-ed77-45c5-8900-50616d8a0d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:28.055920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a56b80-ed77-45c5-8900-50616d8a0d20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:28.066602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a56b80-ed77-45c5-8900-50616d8a0d20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-49a56b80-ed77-45c5-8900-50616d8a0d20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:07:28.074558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:28.075692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:28.076763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 630/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:07:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:28.081797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916c7fc-ae34-4f54-b6cc-e5cc07d08083 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:28.100783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:28.184293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916c7fc-ae34-4f54-b6cc-e5cc07d08083 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:28.219552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd866aa8-0e4e-4161-ad81-936d8026b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:28.220223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd866aa8-0e4e-4161-ad81-936d8026b4af None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:28.220507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd866aa8-0e4e-4161-ad81-936d8026b4af None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:28.220779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd866aa8-0e4e-4161-ad81-936d8026b4af None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:28.221524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd866aa8-0e4e-4161-ad81-936d8026b4af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:28.221863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd866aa8-0e4e-4161-ad81-936d8026b4af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:28.223253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd866aa8-0e4e-4161-ad81-936d8026b4af None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:28.231961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 634/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:07:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:28.242149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a56b80-ed77-45c5-8900-50616d8a0d20 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:28.342025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:28.390637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a56b80-ed77-45c5-8900-50616d8a0d20 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:28.430620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f53322-c795-487d-a3b1-c8004576c8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:28.431401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f53322-c795-487d-a3b1-c8004576c8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:28.432290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f53322-c795-487d-a3b1-c8004576c8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:28.432620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f53322-c795-487d-a3b1-c8004576c8f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:28.443424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f53322-c795-487d-a3b1-c8004576c8f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:28.444879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f53322-c795-487d-a3b1-c8004576c8f3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-15f53322-c795-487d-a3b1-c8004576c8f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:28.463217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:28.464424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:28.465719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:28.465719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:28.465719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:28.465719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:28.465719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 631/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:28.470258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd866aa8-0e4e-4161-ad81-936d8026b4af None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:28.556514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:28.579679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd866aa8-0e4e-4161-ad81-936d8026b4af None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:28.615834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0abd6d-6b26-4c7e-9c94-785ccf36ac7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:28.616485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0abd6d-6b26-4c7e-9c94-785ccf36ac7a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:28.616767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0abd6d-6b26-4c7e-9c94-785ccf36ac7a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:28.617037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0abd6d-6b26-4c7e-9c94-785ccf36ac7a None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:28.617772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0abd6d-6b26-4c7e-9c94-785ccf36ac7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:28.618079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0abd6d-6b26-4c7e-9c94-785ccf36ac7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:28.619608 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0abd6d-6b26-4c7e-9c94-785ccf36ac7a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:28.632088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 635/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:28.639069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f53322-c795-487d-a3b1-c8004576c8f3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:28.754496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f53322-c795-487d-a3b1-c8004576c8f3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:28.776158 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:28.791559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89a400-8881-4b2f-bf54-f009c7317ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:28.792372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89a400-8881-4b2f-bf54-f009c7317ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:28.792650 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89a400-8881-4b2f-bf54-f009c7317ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:28.792922 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89a400-8881-4b2f-bf54-f009c7317ea4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:28.800707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89a400-8881-4b2f-bf54-f009c7317ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:28.802194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89a400-8881-4b2f-bf54-f009c7317ea4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-db89a400-8881-4b2f-bf54-f009c7317ea4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:07:28.816794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:28.817967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:28.818936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:28.818936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:28.818936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:07:28.818936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:07:28.818936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:07:28.818936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:07:28.818936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:07:28.818936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:07:28.818936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 18:07:28.818936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:28.818936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 632/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:28.824708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0abd6d-6b26-4c7e-9c94-785ccf36ac7a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:28.925198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0abd6d-6b26-4c7e-9c94-785ccf36ac7a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:28.962480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73e589-17c3-4a89-b474-8a719c326383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:28.963340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73e589-17c3-4a89-b474-8a719c326383 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:28.963678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73e589-17c3-4a89-b474-8a719c326383 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:28.964044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73e589-17c3-4a89-b474-8a719c326383 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:28.964889 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73e589-17c3-4a89-b474-8a719c326383 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:28.965200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73e589-17c3-4a89-b474-8a719c326383 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:28.966677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73e589-17c3-4a89-b474-8a719c326383 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:28.975890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:28.984864 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 636/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Thu May 26 18:07:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:28.990719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89a400-8881-4b2f-bf54-f009c7317ea4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:29.108884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89a400-8881-4b2f-bf54-f009c7317ea4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:29.142916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb1bdb-3ad6-452e-8ed8-ee4c2bb57bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:29.143692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb1bdb-3ad6-452e-8ed8-ee4c2bb57bbb None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:29.143971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb1bdb-3ad6-452e-8ed8-ee4c2bb57bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:29.144239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb1bdb-3ad6-452e-8ed8-ee4c2bb57bbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:29.154843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb1bdb-3ad6-452e-8ed8-ee4c2bb57bbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:29.156280 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb1bdb-3ad6-452e-8ed8-ee4c2bb57bbb None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-2fbb1bdb-3ad6-452e-8ed8-ee4c2bb57bbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:29.174514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:29.175700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:29.181314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:29.181314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:29.181314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:29.181314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:29.181314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 633/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:29.181681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73e589-17c3-4a89-b474-8a719c326383 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:29.295425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73e589-17c3-4a89-b474-8a719c326383 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:29.330560 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9a723-1f6e-49b3-b81a-ee136568d328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:29.331219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9a723-1f6e-49b3-b81a-ee136568d328 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:29.331499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9a723-1f6e-49b3-b81a-ee136568d328 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:29.331770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9a723-1f6e-49b3-b81a-ee136568d328 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:29.333362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9a723-1f6e-49b3-b81a-ee136568d328 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:29.333362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9a723-1f6e-49b3-b81a-ee136568d328 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:29.334549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9a723-1f6e-49b3-b81a-ee136568d328 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:29.346670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 637/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:29.352396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb1bdb-3ad6-452e-8ed8-ee4c2bb57bbb None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:29.387084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:29.469891 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb1bdb-3ad6-452e-8ed8-ee4c2bb57bbb None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:29.515350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3d9d4-13e5-47d7-b311-f435795d25d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:29.516137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3d9d4-13e5-47d7-b311-f435795d25d8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:29.516416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3d9d4-13e5-47d7-b311-f435795d25d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:29.516687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3d9d4-13e5-47d7-b311-f435795d25d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:29.523520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3d9d4-13e5-47d7-b311-f435795d25d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:29.524954 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3d9d4-13e5-47d7-b311-f435795d25d8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-3ad3d9d4-13e5-47d7-b311-f435795d25d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:07:29.540005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:29.541818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:29.542918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:07:29.542918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:07:29.542918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:07:29.542918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:07:29.542918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:07:29.542918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:07:29.542918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:07:29.542918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:29.542918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 634/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:07:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:29.548077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9a723-1f6e-49b3-b81a-ee136568d328 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:29.612172 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:29.658457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9a723-1f6e-49b3-b81a-ee136568d328 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:29.699409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb30d7-e5af-42f0-891e-591681c51b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:29.700070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb30d7-e5af-42f0-891e-591681c51b80 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:29.700348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb30d7-e5af-42f0-891e-591681c51b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:29.700615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb30d7-e5af-42f0-891e-591681c51b80 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:29.701401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb30d7-e5af-42f0-891e-591681c51b80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:29.701702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb30d7-e5af-42f0-891e-591681c51b80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:29.703180 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb30d7-e5af-42f0-891e-591681c51b80 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:29.716015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 638/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:07:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:29.722218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3d9d4-13e5-47d7-b311-f435795d25d8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:29.826427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:29.836172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3d9d4-13e5-47d7-b311-f435795d25d8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:29.874213 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81176d5-3de1-4a4b-942c-5a6beb220750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:29.875111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81176d5-3de1-4a4b-942c-5a6beb220750 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:29.875391 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81176d5-3de1-4a4b-942c-5a6beb220750 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:29.875663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81176d5-3de1-4a4b-942c-5a6beb220750 None admin] PATH_INFO: `/v3/projects/4be4045293754773bb52bbb82546a1db/users/7ee40bcb7905490c9ad3e39e7a93b122/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:29.877709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81176d5-3de1-4a4b-942c-5a6beb220750 None admin] RBAC: Authorizing `identity:create_grant(project_id=4be4045293754773bb52bbb82546a1db, user_id=7ee40bcb7905490c9ad3e39e7a93b122, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:29.879167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81176d5-3de1-4a4b-942c-5a6beb220750 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:29.892668 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 635/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 18:07:29 2022] PUT /identity/v3/projects/4be4045293754773bb52bbb82546a1db/users/7ee40bcb7905490c9ad3e39e7a93b122/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:07:32.142897 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe881-7a80-4a66-9909-e4cd218b1f8c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:32.143494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe881-7a80-4a66-9909-e4cd218b1f8c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:32.143494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe881-7a80-4a66-9909-e4cd218b1f8c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:32.144802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 639/1274] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:07:32.150404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07564e-9ecb-4b22-8d2f-da3f27a44d4a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:32.150961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07564e-9ecb-4b22-8d2f-da3f27a44d4a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:32.151142 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07564e-9ecb-4b22-8d2f-da3f27a44d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:32.168566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ec07564e-9ecb-4b22-8d2f-da3f27a44d4a None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:32.169774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ec07564e-9ecb-4b22-8d2f-da3f27a44d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:32.237947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 636/1275] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:07:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:07:32.262510 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfe881-7a80-4a66-9909-e4cd218b1f8c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:32.381682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfe881-7a80-4a66-9909-e4cd218b1f8c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:32.400078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:32.418953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd84ef9-1ce8-4534-9fa7-fa599faaaf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:32.419745 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd84ef9-1ce8-4534-9fa7-fa599faaaf3e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:32.420026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd84ef9-1ce8-4534-9fa7-fa599faaaf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:32.420320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd84ef9-1ce8-4534-9fa7-fa599faaaf3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:32.430568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd84ef9-1ce8-4534-9fa7-fa599faaaf3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-abd84ef9-1ce8-4534-9fa7-fa599faaaf3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 18:07:32.438853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:32.440033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:32.441197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:32.441197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:32.441197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:32.441197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:32.441197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 18:07:32.441197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 18:07:32.441197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 18:07:32.441197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 18:07:32.441197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 18:07:32.441197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 18:07:32.441197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 18:07:32.441197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:32.445280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 640/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 18:07:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:32.446864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07564e-9ecb-4b22-8d2f-da3f27a44d4a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:32.534376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:32.556098 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07564e-9ecb-4b22-8d2f-da3f27a44d4a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:32.590443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9339da-8e5c-4d00-9812-531d51a855aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:32.591112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9339da-8e5c-4d00-9812-531d51a855aa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:32.591394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9339da-8e5c-4d00-9812-531d51a855aa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:32.591663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9339da-8e5c-4d00-9812-531d51a855aa None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:32.592392 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9339da-8e5c-4d00-9812-531d51a855aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:32.592694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9339da-8e5c-4d00-9812-531d51a855aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:32.594083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9339da-8e5c-4d00-9812-531d51a855aa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:32.602405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 637/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 18:07:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:32.609054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd84ef9-1ce8-4534-9fa7-fa599faaaf3e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:32.713419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd84ef9-1ce8-4534-9fa7-fa599faaaf3e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:32.748373 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919fa70-0fc3-4b58-919f-9fe6d68dc96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:32.749675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919fa70-0fc3-4b58-919f-9fe6d68dc96d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:32.750193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919fa70-0fc3-4b58-919f-9fe6d68dc96d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:32.750474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919fa70-0fc3-4b58-919f-9fe6d68dc96d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:32.762285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919fa70-0fc3-4b58-919f-9fe6d68dc96d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:32.763763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919fa70-0fc3-4b58-919f-9fe6d68dc96d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-0919fa70-0fc3-4b58-919f-9fe6d68dc96d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:32.781813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:32.782931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:32.784190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:32.784190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:32.784190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:32.784190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:32.784190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 641/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:32.788735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9339da-8e5c-4d00-9812-531d51a855aa None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:32.896345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9339da-8e5c-4d00-9812-531d51a855aa None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:32.931286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635955a-352b-4266-bf5b-7a8f666bb003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:32.932215 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635955a-352b-4266-bf5b-7a8f666bb003 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:32.932525 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635955a-352b-4266-bf5b-7a8f666bb003 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:32.932798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635955a-352b-4266-bf5b-7a8f666bb003 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:32.933509 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635955a-352b-4266-bf5b-7a8f666bb003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:32.933843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635955a-352b-4266-bf5b-7a8f666bb003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:32.935243 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635955a-352b-4266-bf5b-7a8f666bb003 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:32.948849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 638/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:32.954621 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919fa70-0fc3-4b58-919f-9fe6d68dc96d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:32.986909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:33.085479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919fa70-0fc3-4b58-919f-9fe6d68dc96d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:33.124551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58127375-50db-4a2c-ae24-42fa9439283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:33.125451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58127375-50db-4a2c-ae24-42fa9439283c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:33.125760 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58127375-50db-4a2c-ae24-42fa9439283c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:33.126029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58127375-50db-4a2c-ae24-42fa9439283c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:33.133892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58127375-50db-4a2c-ae24-42fa9439283c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:33.136102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58127375-50db-4a2c-ae24-42fa9439283c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-58127375-50db-4a2c-ae24-42fa9439283c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 18:07:33.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:33.162235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:33.163363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:33.163363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:33.163363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 18:07:33.163363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 18:07:33.163363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 18:07:33.163363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 18:07:33.163363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 18:07:33.163363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 18:07:33.163363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 18:07:33.163363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:33.163363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 642/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:33.169309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635955a-352b-4266-bf5b-7a8f666bb003 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:33.173375 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:33.277552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635955a-352b-4266-bf5b-7a8f666bb003 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:33.314128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc303d6-e5e5-4070-a69c-b5b160b9e922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:33.314816 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc303d6-e5e5-4070-a69c-b5b160b9e922 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:33.315092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc303d6-e5e5-4070-a69c-b5b160b9e922 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:33.315366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc303d6-e5e5-4070-a69c-b5b160b9e922 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:33.316199 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc303d6-e5e5-4070-a69c-b5b160b9e922 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:33.321799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc303d6-e5e5-4070-a69c-b5b160b9e922 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:33.321799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc303d6-e5e5-4070-a69c-b5b160b9e922 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:33.340952 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 639/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Thu May 26 18:07:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:33.346733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58127375-50db-4a2c-ae24-42fa9439283c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:33.449939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:33.459925 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58127375-50db-4a2c-ae24-42fa9439283c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:33.495815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f0160-9e68-41e6-8f55-e671a771466f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:33.496459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f0160-9e68-41e6-8f55-e671a771466f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:33.496740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f0160-9e68-41e6-8f55-e671a771466f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:33.497010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f0160-9e68-41e6-8f55-e671a771466f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:33.577170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f0160-9e68-41e6-8f55-e671a771466f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:33.578404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f0160-9e68-41e6-8f55-e671a771466f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:33.675250 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:33.686249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 643/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 18:07:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 18:07:33.692065 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc303d6-e5e5-4070-a69c-b5b160b9e922 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:33.761381 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:33.801246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc303d6-e5e5-4070-a69c-b5b160b9e922 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:33.835944 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d154f01-5997-4c0f-9079-d15f2378726a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:33.836648 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d154f01-5997-4c0f-9079-d15f2378726a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:33.836922 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d154f01-5997-4c0f-9079-d15f2378726a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:33.837184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d154f01-5997-4c0f-9079-d15f2378726a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:33.848585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d154f01-5997-4c0f-9079-d15f2378726a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:33.850057 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d154f01-5997-4c0f-9079-d15f2378726a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-9d154f01-5997-4c0f-9079-d15f2378726a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 18:07:33.867834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:33.869143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:33.870644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 18:07:33.870644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 18:07:33.870644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 18:07:33.870644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:33.870644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 640/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:33.875244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f0160-9e68-41e6-8f55-e671a771466f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:33.978838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f0160-9e68-41e6-8f55-e671a771466f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:34.017112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f70eae-47e7-4256-8b69-606892ffa3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:34.017799 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f70eae-47e7-4256-8b69-606892ffa3da None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:34.018082 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f70eae-47e7-4256-8b69-606892ffa3da None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:34.018353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f70eae-47e7-4256-8b69-606892ffa3da None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:34.019066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f70eae-47e7-4256-8b69-606892ffa3da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:34.019369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f70eae-47e7-4256-8b69-606892ffa3da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:34.020772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f70eae-47e7-4256-8b69-606892ffa3da None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:34.032704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 644/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 18:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:34.038298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d154f01-5997-4c0f-9079-d15f2378726a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:34.151307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d154f01-5997-4c0f-9079-d15f2378726a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:34.166097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:34.188589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f17e6-aa15-46c7-a3ca-644c5c721349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:34.189390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f17e6-aa15-46c7-a3ca-644c5c721349 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:34.189670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f17e6-aa15-46c7-a3ca-644c5c721349 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:34.189971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f17e6-aa15-46c7-a3ca-644c5c721349 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:34.202614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f17e6-aa15-46c7-a3ca-644c5c721349 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:34.204207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f17e6-aa15-46c7-a3ca-644c5c721349 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-fe5f17e6-aa15-46c7-a3ca-644c5c721349 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 18:07:34.219030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:34.220325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:34.221554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 18:07:34.221554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 18:07:34.221554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 18:07:34.221554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 18:07:34.221554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 18:07:34.221554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 18:07:34.221554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 18:07:34.221554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application May 26 18:07:34.221554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 641/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 18:07:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:34.226344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f70eae-47e7-4256-8b69-606892ffa3da None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:34.279924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:34.336006 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f70eae-47e7-4256-8b69-606892ffa3da None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:34.375172 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f92096-b5e5-455e-89ad-bd23cb2c08bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:34.375872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f92096-b5e5-455e-89ad-bd23cb2c08bc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:34.376149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f92096-b5e5-455e-89ad-bd23cb2c08bc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:34.376427 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f92096-b5e5-455e-89ad-bd23cb2c08bc None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:34.377318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f92096-b5e5-455e-89ad-bd23cb2c08bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:34.377657 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f92096-b5e5-455e-89ad-bd23cb2c08bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:34.379318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f92096-b5e5-455e-89ad-bd23cb2c08bc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:34.393602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 645/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 18:07:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:34.400499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f17e6-aa15-46c7-a3ca-644c5c721349 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:34.509588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f17e6-aa15-46c7-a3ca-644c5c721349 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:34.545497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cfc5e-b663-4682-9d93-9f48ce69cd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:34.546191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cfc5e-b663-4682-9d93-9f48ce69cd51 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:34.546478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cfc5e-b663-4682-9d93-9f48ce69cd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:34.546751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cfc5e-b663-4682-9d93-9f48ce69cd51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:34.547575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cfc5e-b663-4682-9d93-9f48ce69cd51 None admin] RBAC: Adding query filter params (role.id=584beecb022b422f820e77f7623ada5e, scope.project.id=4be4045293754773bb52bbb82546a1db, user.id=7ee40bcb7905490c9ad3e39e7a93b122) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:34.547885 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cfc5e-b663-4682-9d93-9f48ce69cd51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:34.549316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cfc5e-b663-4682-9d93-9f48ce69cd51 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:34.559478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 642/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 18:07:34 2022] GET /identity/v3/role_assignments?user.id=7ee40bcb7905490c9ad3e39e7a93b122&scope.project.id=4be4045293754773bb52bbb82546a1db&role.id=584beecb022b422f820e77f7623ada5e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:36.818884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95185868-950e-4e9a-948a-c21b1fe5a92f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:36.819445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95185868-950e-4e9a-948a-c21b1fe5a92f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:36.819445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95185868-950e-4e9a-948a-c21b1fe5a92f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:36.820795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 646/1288] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:07:36.826431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22abe5f-edd4-4972-9afe-2fef203a6e1b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:36.826935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22abe5f-edd4-4972-9afe-2fef203a6e1b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:36.827027 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22abe5f-edd4-4972-9afe-2fef203a6e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:36.844977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e22abe5f-edd4-4972-9afe-2fef203a6e1b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:36.846201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e22abe5f-edd4-4972-9afe-2fef203a6e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:36.878262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:36.918325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 643/1289] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:07:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:07:36.941177 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95185868-950e-4e9a-948a-c21b1fe5a92f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:37.049890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95185868-950e-4e9a-948a-c21b1fe5a92f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:37.063225 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:37.102493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ad625-db75-4d0e-a61b-5f601245868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:37.104545 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ad625-db75-4d0e-a61b-5f601245868a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:37.105241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ad625-db75-4d0e-a61b-5f601245868a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:37.105720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ad625-db75-4d0e-a61b-5f601245868a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:37.106789 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ad625-db75-4d0e-a61b-5f601245868a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:37.109030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ad625-db75-4d0e-a61b-5f601245868a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-e72ad625-db75-4d0e-a61b-5f601245868a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:37.128648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:37.130618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:37.132480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 647/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 18:07:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:37.136962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22abe5f-edd4-4972-9afe-2fef203a6e1b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:37.259523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22abe5f-edd4-4972-9afe-2fef203a6e1b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:37.302639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f7621-ea6b-4db5-8bcb-fa5c14cb9b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:37.303305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f7621-ea6b-4db5-8bcb-fa5c14cb9b73 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:37.303586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f7621-ea6b-4db5-8bcb-fa5c14cb9b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:37.303855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f7621-ea6b-4db5-8bcb-fa5c14cb9b73 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:37.304559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f7621-ea6b-4db5-8bcb-fa5c14cb9b73 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:37.304859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f7621-ea6b-4db5-8bcb-fa5c14cb9b73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:37.306266 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f7621-ea6b-4db5-8bcb-fa5c14cb9b73 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:37.311947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 644/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Thu May 26 18:07:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:37.320284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ad625-db75-4d0e-a61b-5f601245868a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:37.433065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ad625-db75-4d0e-a61b-5f601245868a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:37.467821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bed5bb-2ba8-4230-a7aa-d291af157c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:37.468483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bed5bb-2ba8-4230-a7aa-d291af157c42 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:37.468768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bed5bb-2ba8-4230-a7aa-d291af157c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:37.469046 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bed5bb-2ba8-4230-a7aa-d291af157c42 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:37.469779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bed5bb-2ba8-4230-a7aa-d291af157c42 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:37.470091 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bed5bb-2ba8-4230-a7aa-d291af157c42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:37.471481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bed5bb-2ba8-4230-a7aa-d291af157c42 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:37.476444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 648/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Thu May 26 18:07:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:39.709336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab65a5-eeff-459b-804d-d204101f709d None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:39.709934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab65a5-eeff-459b-804d-d204101f709d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:39.709934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab65a5-eeff-459b-804d-d204101f709d None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:39.711275 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 645/1293] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:07:39.717417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7e0c9-295d-4a3b-aee9-677f05474d55 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:39.717965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7e0c9-295d-4a3b-aee9-677f05474d55 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:39.717965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7e0c9-295d-4a3b-aee9-677f05474d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:39.734972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e3a7e0c9-295d-4a3b-aee9-677f05474d55 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:39.736176 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e3a7e0c9-295d-4a3b-aee9-677f05474d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:39.767416 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:39.805942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 649/1294] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:07:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:07:39.829108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab65a5-eeff-459b-804d-d204101f709d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:39.861140 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:39.934558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab65a5-eeff-459b-804d-d204101f709d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:39.969492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c934ed-9c38-4e33-b4b5-dc865c4f82ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:39.970095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c934ed-9c38-4e33-b4b5-dc865c4f82ac None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:39.970377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c934ed-9c38-4e33-b4b5-dc865c4f82ac None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:39.970648 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c934ed-9c38-4e33-b4b5-dc865c4f82ac None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:39.971376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c934ed-9c38-4e33-b4b5-dc865c4f82ac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:39.972741 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c934ed-9c38-4e33-b4b5-dc865c4f82ac None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:39.987821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 646/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 18:07:39 2022] POST /identity/v3/services => generated 246 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:42.262186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d066c0b-a500-4ef1-ba2c-f23907fc93d0 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.262714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d066c0b-a500-4ef1-ba2c-f23907fc93d0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.262714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d066c0b-a500-4ef1-ba2c-f23907fc93d0 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.264091 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 650/1296] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:07:42.269515 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b043cc-aab8-4bf7-90bf-4b5b40846c1e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.269990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b043cc-aab8-4bf7-90bf-4b5b40846c1e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.270091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b043cc-aab8-4bf7-90bf-4b5b40846c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.287551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a5b043cc-aab8-4bf7-90bf-4b5b40846c1e None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:42.288758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a5b043cc-aab8-4bf7-90bf-4b5b40846c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:42.365586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 647/1297] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:07:42 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:07:42.392366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d066c0b-a500-4ef1-ba2c-f23907fc93d0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:42.503739 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d066c0b-a500-4ef1-ba2c-f23907fc93d0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:42.540074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7930676-1119-43ba-8c2f-0e4b0d0f1618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:42.540814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7930676-1119-43ba-8c2f-0e4b0d0f1618 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.541094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7930676-1119-43ba-8c2f-0e4b0d0f1618 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.541364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7930676-1119-43ba-8c2f-0e4b0d0f1618 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.542045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7930676-1119-43ba-8c2f-0e4b0d0f1618 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:42.543513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7930676-1119-43ba-8c2f-0e4b0d0f1618 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c7930676-1119-43ba-8c2f-0e4b0d0f1618 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:42.555543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:42.557290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:42.558669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 651/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 18:07:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:42.563419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b043cc-aab8-4bf7-90bf-4b5b40846c1e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:42.582849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:42.672372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b043cc-aab8-4bf7-90bf-4b5b40846c1e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:42.714777 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb3def-bbbc-4003-b229-c8fdfa786a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:42.715442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb3def-bbbc-4003-b229-c8fdfa786a42 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.715754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb3def-bbbc-4003-b229-c8fdfa786a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.716035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb3def-bbbc-4003-b229-c8fdfa786a42 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.716736 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb3def-bbbc-4003-b229-c8fdfa786a42 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:42.717042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb3def-bbbc-4003-b229-c8fdfa786a42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:42.718475 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb3def-bbbc-4003-b229-c8fdfa786a42 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:42.723999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 648/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Thu May 26 18:07:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:07:42.730411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7930676-1119-43ba-8c2f-0e4b0d0f1618 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:42.763606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:42.835745 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7930676-1119-43ba-8c2f-0e4b0d0f1618 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:42.871161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053aa64-80fe-4990-8e5b-7f47fdd78559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:42.872294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053aa64-80fe-4990-8e5b-7f47fdd78559 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.872781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053aa64-80fe-4990-8e5b-7f47fdd78559 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.873125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053aa64-80fe-4990-8e5b-7f47fdd78559 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.873940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053aa64-80fe-4990-8e5b-7f47fdd78559 None admin] RBAC: Adding query filter params (interface=public, service_id=39f0c0e5a91d471cb334f4fa4d020ac1, region_id=RegionOne) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:42.874255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053aa64-80fe-4990-8e5b-7f47fdd78559 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:42.876379 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053aa64-80fe-4990-8e5b-7f47fdd78559 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:42.881684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 652/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 18:07:42 2022] GET /identity/v3/endpoints?service_id=39f0c0e5a91d471cb334f4fa4d020ac1&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:42.888182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb3def-bbbc-4003-b229-c8fdfa786a42 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:42.983295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:43.003339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb3def-bbbc-4003-b229-c8fdfa786a42 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:43.042537 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ecec88-4001-4d27-a4ea-7b82f8e10e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:43.043617 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ecec88-4001-4d27-a4ea-7b82f8e10e76 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:43.044077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ecec88-4001-4d27-a4ea-7b82f8e10e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:43.044571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ecec88-4001-4d27-a4ea-7b82f8e10e76 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:43.045755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ecec88-4001-4d27-a4ea-7b82f8e10e76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:43.048165 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ecec88-4001-4d27-a4ea-7b82f8e10e76 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:43.061518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 649/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 18:07:42 2022] GET /identity/v3/services => generated 2191 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:45.304233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9fb1-b11e-4071-a12c-5fdb99543a6c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:45.304768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9fb1-b11e-4071-a12c-5fdb99543a6c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:45.304768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9fb1-b11e-4071-a12c-5fdb99543a6c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:45.306181 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 653/1302] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:07:45.312247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3430d2-813b-4cde-895a-c5cef71d7dbe None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:45.312728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3430d2-813b-4cde-895a-c5cef71d7dbe None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:45.312837 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3430d2-813b-4cde-895a-c5cef71d7dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:45.331492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-8f3430d2-813b-4cde-895a-c5cef71d7dbe None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:45.332688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8f3430d2-813b-4cde-895a-c5cef71d7dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:45.402755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 650/1303] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:07:45 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 18:07:45.425478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e9fb1-b11e-4071-a12c-5fdb99543a6c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:45.525620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:45.533092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e9fb1-b11e-4071-a12c-5fdb99543a6c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:45.568921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd0729-a4f4-4081-b9eb-79550392871f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:45.569641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd0729-a4f4-4081-b9eb-79550392871f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:45.569952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd0729-a4f4-4081-b9eb-79550392871f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:45.570226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd0729-a4f4-4081-b9eb-79550392871f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:45.570878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd0729-a4f4-4081-b9eb-79550392871f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:45.572817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd0729-a4f4-4081-b9eb-79550392871f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-17fd0729-a4f4-4081-b9eb-79550392871f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 18:07:45.585010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 18:07:45.586211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application May 26 18:07:45.587190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 654/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 18:07:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 18:07:45.595829 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3430d2-813b-4cde-895a-c5cef71d7dbe None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:45.675609 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:45.699683 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3430d2-813b-4cde-895a-c5cef71d7dbe None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:45.734974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7a767-8dfe-444e-b16c-f0b8c57feab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:45.735638 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7a767-8dfe-444e-b16c-f0b8c57feab8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:45.735916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7a767-8dfe-444e-b16c-f0b8c57feab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:45.736186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7a767-8dfe-444e-b16c-f0b8c57feab8 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:45.736881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7a767-8dfe-444e-b16c-f0b8c57feab8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:45.737185 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7a767-8dfe-444e-b16c-f0b8c57feab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:45.738579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7a767-8dfe-444e-b16c-f0b8c57feab8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:45.743968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 651/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Thu May 26 18:07:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:45.750726 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd0729-a4f4-4081-b9eb-79550392871f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:45.856123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd0729-a4f4-4081-b9eb-79550392871f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:45.892934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2aa7ce-59a2-4466-ad45-e5a034d92373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:45.893590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2aa7ce-59a2-4466-ad45-e5a034d92373 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:45.893905 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2aa7ce-59a2-4466-ad45-e5a034d92373 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:45.894217 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2aa7ce-59a2-4466-ad45-e5a034d92373 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:45.894961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2aa7ce-59a2-4466-ad45-e5a034d92373 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:45.896354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2aa7ce-59a2-4466-ad45-e5a034d92373 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:45.916298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 655/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 18:07:45 2022] POST /identity/v3/endpoints => generated 331 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:55.741125 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=30 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:55.910910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:53.363682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28824bf-b67f-4d31-b19b-a6e2fe5e68af None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:53.364768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28824bf-b67f-4d31-b19b-a6e2fe5e68af None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:53.365338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28824bf-b67f-4d31-b19b-a6e2fe5e68af None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:53.367974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 652/1307] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:08:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:08:53.373456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534e340-0ca5-4d59-8e5c-054462aa4b2c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:53.373914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534e340-0ca5-4d59-8e5c-054462aa4b2c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:53.374115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534e340-0ca5-4d59-8e5c-054462aa4b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:53.506185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6534e340-0ca5-4d59-8e5c-054462aa4b2c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:08:53.510864 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6534e340-0ca5-4d59-8e5c-054462aa4b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:53.668748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 656/1308] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:08:53.727115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b036e26-82dc-43ad-854e-71a1df9d11cd None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:53.727395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b036e26-82dc-43ad-854e-71a1df9d11cd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:53.727929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b036e26-82dc-43ad-854e-71a1df9d11cd None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:53.729328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 653/1309] 10.222.0.46 () {54 vars in 964 bytes} [Thu May 26 18:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:08:53.736079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67125dbb-f845-414e-9256-fc6b7dc24f9c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:53.736449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67125dbb-f845-414e-9256-fc6b7dc24f9c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:53.736843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67125dbb-f845-414e-9256-fc6b7dc24f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:53.819562 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-67125dbb-f845-414e-9256-fc6b7dc24f9c None None] MFA Rules not processed for user `f668c4c6745e4f1eae20d8a3a12cf0cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:08:53.821320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-67125dbb-f845-414e-9256-fc6b7dc24f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:53.842960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:53.905241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 657/1310] 10.222.0.46 () {58 vars in 1061 bytes} [Thu May 26 18:08:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:08:53.914706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b036e26-82dc-43ad-854e-71a1df9d11cd None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:53.917049 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4b036e26-82dc-43ad-854e-71a1df9d11cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:54.041251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b036e26-82dc-43ad-854e-71a1df9d11cd None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:54.121653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788bf35-7242-4162-91b1-0003794f8872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:54.122241 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788bf35-7242-4162-91b1-0003794f8872 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:54.122516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788bf35-7242-4162-91b1-0003794f8872 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:54.122780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788bf35-7242-4162-91b1-0003794f8872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:54.124439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4788bf35-7242-4162-91b1-0003794f8872 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:54.189296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:54.254672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788bf35-7242-4162-91b1-0003794f8872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:54.256122 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788bf35-7242-4162-91b1-0003794f8872 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:54.278518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 654/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:08:56.847283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcebf8-d72a-471e-ac70-c5cd7c25f128 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:56.847790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcebf8-d72a-471e-ac70-c5cd7c25f128 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:56.847790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcebf8-d72a-471e-ac70-c5cd7c25f128 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:56.849414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 658/1312] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:08:56.855868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676975f-0605-40bc-9627-4f19a48de53f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:56.856434 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676975f-0605-40bc-9627-4f19a48de53f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:56.856434 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676975f-0605-40bc-9627-4f19a48de53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:56.951521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1676975f-0605-40bc-9627-4f19a48de53f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:08:56.952734 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1676975f-0605-40bc-9627-4f19a48de53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:57.058405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 655/1313] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:08:57.097378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bcebf8-d72a-471e-ac70-c5cd7c25f128 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:57.099277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b7bcebf8-d72a-471e-ac70-c5cd7c25f128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:57.213223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bcebf8-d72a-471e-ac70-c5cd7c25f128 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:57.299993 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:57.302886 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a43698-972b-481e-a9a6-365bf6b9e385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:57.303597 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a43698-972b-481e-a9a6-365bf6b9e385 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:57.303896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a43698-972b-481e-a9a6-365bf6b9e385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:57.304152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a43698-972b-481e-a9a6-365bf6b9e385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:57.307306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f1a43698-972b-481e-a9a6-365bf6b9e385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:57.445859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a43698-972b-481e-a9a6-365bf6b9e385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:57.447670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a43698-972b-481e-a9a6-365bf6b9e385 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:57.470336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 659/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:00.116017 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ebcbd-12cf-40f6-81ad-ae486778de07 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:00.116589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ebcbd-12cf-40f6-81ad-ae486778de07 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:00.116589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ebcbd-12cf-40f6-81ad-ae486778de07 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:00.118719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 656/1315] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:00.124016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69aa5c-82bd-4003-841f-4f6e3f3c2ab5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:00.124446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69aa5c-82bd-4003-841f-4f6e3f3c2ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:00.124554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69aa5c-82bd-4003-841f-4f6e3f3c2ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:00.194389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-cd69aa5c-82bd-4003-841f-4f6e3f3c2ab5 None None] MFA Rules not processed for user `331362e99eb14800b4d52472fd41a629`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:00.196164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cd69aa5c-82bd-4003-841f-4f6e3f3c2ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:00.247944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:00.293455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 660/1316] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:00.333368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ebcbd-12cf-40f6-81ad-ae486778de07 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:00.334959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-965ebcbd-12cf-40f6-81ad-ae486778de07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:00.344906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:00.438996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ebcbd-12cf-40f6-81ad-ae486778de07 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:00.513601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029d0f4-4aff-4190-8a51-380a13d5c075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:00.514293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029d0f4-4aff-4190-8a51-380a13d5c075 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:00.514559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029d0f4-4aff-4190-8a51-380a13d5c075 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:00.514831 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029d0f4-4aff-4190-8a51-380a13d5c075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:00.516488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f029d0f4-4aff-4190-8a51-380a13d5c075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:00.578298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:00.624934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029d0f4-4aff-4190-8a51-380a13d5c075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:00.626404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029d0f4-4aff-4190-8a51-380a13d5c075 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:00.645057 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 657/1317] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:03.451833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5403d0-1e51-421e-99a8-0c270cd2b3ca None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:03.452334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5403d0-1e51-421e-99a8-0c270cd2b3ca None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:03.452334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5403d0-1e51-421e-99a8-0c270cd2b3ca None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:03.453695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 661/1318] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:03.458855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bff41-a75a-41d5-9920-67c6b1a9f290 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:03.459266 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bff41-a75a-41d5-9920-67c6b1a9f290 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:03.459402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bff41-a75a-41d5-9920-67c6b1a9f290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:03.527716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-597bff41-a75a-41d5-9920-67c6b1a9f290 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:03.528918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-597bff41-a75a-41d5-9920-67c6b1a9f290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:03.630602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 658/1319] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:03.670059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5403d0-1e51-421e-99a8-0c270cd2b3ca None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:03.671587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7d5403d0-1e51-421e-99a8-0c270cd2b3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:03.780078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5403d0-1e51-421e-99a8-0c270cd2b3ca None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:03.853037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70779ff-5b0d-4b3d-a593-de039d649344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:03.853672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70779ff-5b0d-4b3d-a593-de039d649344 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:03.853978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70779ff-5b0d-4b3d-a593-de039d649344 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:03.854252 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70779ff-5b0d-4b3d-a593-de039d649344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:03.855883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a70779ff-5b0d-4b3d-a593-de039d649344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:03.863224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:03.978069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70779ff-5b0d-4b3d-a593-de039d649344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:03.979544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70779ff-5b0d-4b3d-a593-de039d649344 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:03.999868 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 662/1320] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:06.934726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01906e8-06d5-49d6-8a86-e274af35c3b9 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:06.935300 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01906e8-06d5-49d6-8a86-e274af35c3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:06.935300 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01906e8-06d5-49d6-8a86-e274af35c3b9 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:06.936663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 659/1321] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:06.942576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a10e80-7221-4530-9c0f-f54913300ab6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:06.943019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a10e80-7221-4530-9c0f-f54913300ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:06.943249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a10e80-7221-4530-9c0f-f54913300ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:07.012575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-94a10e80-7221-4530-9c0f-f54913300ab6 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:07.013819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-94a10e80-7221-4530-9c0f-f54913300ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:07.076814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:07.121753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 663/1322] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:07.157867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01906e8-06d5-49d6-8a86-e274af35c3b9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:07.159456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a01906e8-06d5-49d6-8a86-e274af35c3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:07.188907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:07.284605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01906e8-06d5-49d6-8a86-e274af35c3b9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:07.366428 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebf636-7838-491f-920e-985dc54bfc08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:07.367786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebf636-7838-491f-920e-985dc54bfc08 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:07.368286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebf636-7838-491f-920e-985dc54bfc08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:07.368587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebf636-7838-491f-920e-985dc54bfc08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:07.370331 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-46ebf636-7838-491f-920e-985dc54bfc08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:07.502072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:07.568566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebf636-7838-491f-920e-985dc54bfc08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:07.571810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebf636-7838-491f-920e-985dc54bfc08 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:07.603094 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 660/1323] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:10.019931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc3949-ae62-45fb-afba-633ee19188f3 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:10.020579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc3949-ae62-45fb-afba-633ee19188f3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:10.020944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc3949-ae62-45fb-afba-633ee19188f3 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:10.023717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 664/1324] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:10.032815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116f442-ca5e-4e83-8689-52ed9478d483 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:10.033432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116f442-ca5e-4e83-8689-52ed9478d483 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:10.033852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116f442-ca5e-4e83-8689-52ed9478d483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:10.106924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9116f442-ca5e-4e83-8689-52ed9478d483 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:10.108156 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9116f442-ca5e-4e83-8689-52ed9478d483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:10.210135 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 661/1325] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:10.259224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d15bd-b8f3-4757-930c-53db9be30381 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:10.259486 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d15bd-b8f3-4757-930c-53db9be30381 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:10.259765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d15bd-b8f3-4757-930c-53db9be30381 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:10.261124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 665/1326] 10.222.0.46 () {54 vars in 964 bytes} [Thu May 26 18:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:09:10.267166 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d1be3-b723-404e-8988-a3f0ecf05b1a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:10.267537 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d1be3-b723-404e-8988-a3f0ecf05b1a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:10.267884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d1be3-b723-404e-8988-a3f0ecf05b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:10.300628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:10.354046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-864d1be3-b723-404e-8988-a3f0ecf05b1a None None] MFA Rules not processed for user `f668c4c6745e4f1eae20d8a3a12cf0cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:10.355242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-864d1be3-b723-404e-8988-a3f0ecf05b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:10.433843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 662/1327] 10.222.0.46 () {58 vars in 1061 bytes} [Thu May 26 18:09:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:10.442977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d15bd-b8f3-4757-930c-53db9be30381 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:10.444492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-410d15bd-b8f3-4757-930c-53db9be30381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:10.549677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d15bd-b8f3-4757-930c-53db9be30381 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:10.625473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345705a-ce73-4691-bd2f-e4c64c454ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:10.626068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345705a-ce73-4691-bd2f-e4c64c454ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:10.626349 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345705a-ce73-4691-bd2f-e4c64c454ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:10.626617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345705a-ce73-4691-bd2f-e4c64c454ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:10.628224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7345705a-ce73-4691-bd2f-e4c64c454ea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:10.646589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:10.743481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345705a-ce73-4691-bd2f-e4c64c454ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:10.744930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345705a-ce73-4691-bd2f-e4c64c454ea4 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:10.763219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 666/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:13.330737 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee907d-72e1-485b-bd32-b58e10201008 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:13.331215 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee907d-72e1-485b-bd32-b58e10201008 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:13.331316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee907d-72e1-485b-bd32-b58e10201008 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:13.332658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 663/1329] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:13.338757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77600008-ceb3-44e4-8515-cdc432672b63 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:13.339218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77600008-ceb3-44e4-8515-cdc432672b63 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:13.339321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77600008-ceb3-44e4-8515-cdc432672b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:13.405903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-77600008-ceb3-44e4-8515-cdc432672b63 None None] MFA Rules not processed for user `331362e99eb14800b4d52472fd41a629`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:13.407107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-77600008-ceb3-44e4-8515-cdc432672b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:13.471674 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:13.503435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 667/1330] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:13 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:13.541225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee907d-72e1-485b-bd32-b58e10201008 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:13.542778 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1eee907d-72e1-485b-bd32-b58e10201008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:13.647004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee907d-72e1-485b-bd32-b58e10201008 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:13.668311 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:13.721318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e3239-ef33-4222-878b-158bfa171d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:13.722041 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e3239-ef33-4222-878b-158bfa171d51 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:13.722272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e3239-ef33-4222-878b-158bfa171d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:13.722544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e3239-ef33-4222-878b-158bfa171d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:13.724156 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-939e3239-ef33-4222-878b-158bfa171d51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:13.833128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e3239-ef33-4222-878b-158bfa171d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:13.834594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e3239-ef33-4222-878b-158bfa171d51 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:13.854800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 664/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:16.570340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac993926-5a66-46d7-88eb-08a92ecda4ae None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:16.570853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac993926-5a66-46d7-88eb-08a92ecda4ae None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:16.571165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac993926-5a66-46d7-88eb-08a92ecda4ae None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:16.572588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 668/1332] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:16.578246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8735d6-4d93-4eab-b9de-dc16065028d8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:16.578727 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8735d6-4d93-4eab-b9de-dc16065028d8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:16.578830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8735d6-4d93-4eab-b9de-dc16065028d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:16.646704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:16.650542 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0f8735d6-4d93-4eab-b9de-dc16065028d8 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:16.652265 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0f8735d6-4d93-4eab-b9de-dc16065028d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:16.751960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 665/1333] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:16.789937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac993926-5a66-46d7-88eb-08a92ecda4ae None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:16.791783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ac993926-5a66-46d7-88eb-08a92ecda4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:16.920653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac993926-5a66-46d7-88eb-08a92ecda4ae None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:16.996545 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59528877-05e0-403f-9b22-b3e873cf214e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:16.997606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59528877-05e0-403f-9b22-b3e873cf214e service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:16.998111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59528877-05e0-403f-9b22-b3e873cf214e service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:16.998577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59528877-05e0-403f-9b22-b3e873cf214e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:17.001598 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-59528877-05e0-403f-9b22-b3e873cf214e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:17.032686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:17.115259 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59528877-05e0-403f-9b22-b3e873cf214e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:17.117030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59528877-05e0-403f-9b22-b3e873cf214e service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:17.135987 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 669/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:19.447901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf5d7f-0152-4098-8a35-255cc835d9d3 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:19.448450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf5d7f-0152-4098-8a35-255cc835d9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:19.448450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf5d7f-0152-4098-8a35-255cc835d9d3 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:19.449750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 666/1335] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:19.455034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43736e0-1d94-4a0d-a3a2-6afda688722d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:19.455418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43736e0-1d94-4a0d-a3a2-6afda688722d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:19.455587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43736e0-1d94-4a0d-a3a2-6afda688722d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:19.519494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e43736e0-1d94-4a0d-a3a2-6afda688722d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:19.520689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e43736e0-1d94-4a0d-a3a2-6afda688722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:19.594684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:19.624011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 670/1336] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:19.662910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf5d7f-0152-4098-8a35-255cc835d9d3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:19.664962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3dcf5d7f-0152-4098-8a35-255cc835d9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:19.766476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf5d7f-0152-4098-8a35-255cc835d9d3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:19.795374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:19.839561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f9a7e-75eb-420b-8927-cf08336a4445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:19.840197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f9a7e-75eb-420b-8927-cf08336a4445 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:19.840475 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f9a7e-75eb-420b-8927-cf08336a4445 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:19.840746 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f9a7e-75eb-420b-8927-cf08336a4445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:19.842391 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d31f9a7e-75eb-420b-8927-cf08336a4445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:19.959526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f9a7e-75eb-420b-8927-cf08336a4445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:19.961538 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f9a7e-75eb-420b-8927-cf08336a4445 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:19.980412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 667/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:22.787176 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddbe87-290e-4efc-b941-386016b03d3c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:22.787746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddbe87-290e-4efc-b941-386016b03d3c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:22.787746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddbe87-290e-4efc-b941-386016b03d3c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:22.789145 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 671/1338] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:22.794713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce21a43-602b-4e7a-bc45-0b154104de6d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:22.795140 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce21a43-602b-4e7a-bc45-0b154104de6d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:22.795271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce21a43-602b-4e7a-bc45-0b154104de6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:22.861601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2ce21a43-602b-4e7a-bc45-0b154104de6d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:22.862806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2ce21a43-602b-4e7a-bc45-0b154104de6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:22.871196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:22.964739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 668/1339] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:22.999935 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddbe87-290e-4efc-b941-386016b03d3c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:23.001456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-05ddbe87-290e-4efc-b941-386016b03d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:23.112299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddbe87-290e-4efc-b941-386016b03d3c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:23.186342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15134f-f1c4-4e09-a64f-aa12f8604d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:23.186991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15134f-f1c4-4e09-a64f-aa12f8604d9d service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:23.187269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15134f-f1c4-4e09-a64f-aa12f8604d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:23.187537 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15134f-f1c4-4e09-a64f-aa12f8604d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:23.189345 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4a15134f-f1c4-4e09-a64f-aa12f8604d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:23.225981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:23.299659 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15134f-f1c4-4e09-a64f-aa12f8604d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:23.301112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15134f-f1c4-4e09-a64f-aa12f8604d9d service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:23.319852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 672/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:27.753216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322d58c-83fe-4ab9-9a70-4d431aea1687 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:27.753762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322d58c-83fe-4ab9-9a70-4d431aea1687 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:27.753762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322d58c-83fe-4ab9-9a70-4d431aea1687 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:27.755169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 669/1341] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:27.761129 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da5185-4097-4053-96cb-8c387036fcc1 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:27.761577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da5185-4097-4053-96cb-8c387036fcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:27.761683 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da5185-4097-4053-96cb-8c387036fcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:27.835929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d8da5185-4097-4053-96cb-8c387036fcc1 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:27.837210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d8da5185-4097-4053-96cb-8c387036fcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:27.935991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:27.950619 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 673/1342] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:27.989467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322d58c-83fe-4ab9-9a70-4d431aea1687 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:27.993379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0322d58c-83fe-4ab9-9a70-4d431aea1687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:28.123795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322d58c-83fe-4ab9-9a70-4d431aea1687 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:28.127291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea13e8-c2b0-4188-8c52-aa6725377c51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:28.127932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea13e8-c2b0-4188-8c52-aa6725377c51 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:28.128211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea13e8-c2b0-4188-8c52-aa6725377c51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:28.128481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea13e8-c2b0-4188-8c52-aa6725377c51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:28.138331 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea13e8-c2b0-4188-8c52-aa6725377c51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:28.139762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea13e8-c2b0-4188-8c52-aa6725377c51 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:28.149189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 670/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:31.015307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2001264-1e86-4639-86ec-1f995777a87c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:31.016130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2001264-1e86-4639-86ec-1f995777a87c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:31.016130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2001264-1e86-4639-86ec-1f995777a87c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:31.017663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 674/1344] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:31.024784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a564381-c7b4-44fd-8efc-99abb6608061 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:31.025236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a564381-c7b4-44fd-8efc-99abb6608061 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:31.025401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a564381-c7b4-44fd-8efc-99abb6608061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:31.067112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4a564381-c7b4-44fd-8efc-99abb6608061 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:31.068326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4a564381-c7b4-44fd-8efc-99abb6608061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:31.098559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:31.147490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 671/1345] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:31.184375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2001264-1e86-4639-86ec-1f995777a87c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:31.195260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2001264-1e86-4639-86ec-1f995777a87c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:31.198771 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab533a-eb21-4f87-ade1-06ed5652eca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:31.199408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab533a-eb21-4f87-ade1-06ed5652eca7 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:31.199683 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab533a-eb21-4f87-ade1-06ed5652eca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:31.199955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab533a-eb21-4f87-ade1-06ed5652eca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:31.210156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab533a-eb21-4f87-ade1-06ed5652eca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:31.212030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab533a-eb21-4f87-ade1-06ed5652eca7 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:31.221705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 675/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:35.476931 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807af45b-ee09-4231-b257-d755be3303be None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:35.477479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807af45b-ee09-4231-b257-d755be3303be None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:35.477576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807af45b-ee09-4231-b257-d755be3303be None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:35.478957 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 672/1347] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:35.484976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606967d7-6ca7-4859-b5bd-161fe6132642 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:35.484976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606967d7-6ca7-4859-b5bd-161fe6132642 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:35.485451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606967d7-6ca7-4859-b5bd-161fe6132642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:35.504349 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-606967d7-6ca7-4859-b5bd-161fe6132642 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:35.505640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-606967d7-6ca7-4859-b5bd-161fe6132642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:35.514426 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 676/1348] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:35.551752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807af45b-ee09-4231-b257-d755be3303be None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:35.554480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-807af45b-ee09-4231-b257-d755be3303be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:35.567553 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807af45b-ee09-4231-b257-d755be3303be None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:35.571821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57e71b-2df1-4845-a12b-ee62469056db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:35.572465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57e71b-2df1-4845-a12b-ee62469056db service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:35.572745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57e71b-2df1-4845-a12b-ee62469056db service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:35.573018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57e71b-2df1-4845-a12b-ee62469056db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:35.582332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57e71b-2df1-4845-a12b-ee62469056db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:35.583756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57e71b-2df1-4845-a12b-ee62469056db service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:35.593929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 673/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:41.074612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12cd0c-755d-467e-a837-3ef42cc6c007 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:41.075499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12cd0c-755d-467e-a837-3ef42cc6c007 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:41.075925 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12cd0c-755d-467e-a837-3ef42cc6c007 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:41.077747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 677/1350] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:09:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:41.083235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af537583-0feb-4339-985c-69b5bd554e90 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:41.084008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af537583-0feb-4339-985c-69b5bd554e90 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:41.084933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af537583-0feb-4339-985c-69b5bd554e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:41.159559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-af537583-0feb-4339-985c-69b5bd554e90 None None] MFA Rules not processed for user `7ee40bcb7905490c9ad3e39e7a93b122`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:41.161130 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-af537583-0feb-4339-985c-69b5bd554e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:41.304057 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 674/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:09:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:45.498719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=13 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:45.746168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c96a8-cf9b-4155-bbb7-a6fafe80b87f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:45.746769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c96a8-cf9b-4155-bbb7-a6fafe80b87f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:45.747226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c96a8-cf9b-4155-bbb7-a6fafe80b87f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:45.748657 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 678/1352] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:09:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:45.754501 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc56ffa-7157-4d2b-9362-fbeffa057c2b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:45.754991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc56ffa-7157-4d2b-9362-fbeffa057c2b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:45.755103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc56ffa-7157-4d2b-9362-fbeffa057c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:45.774018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-acc56ffa-7157-4d2b-9362-fbeffa057c2b None None] MFA Rules not processed for user `7ee40bcb7905490c9ad3e39e7a93b122`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:45.775336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-acc56ffa-7157-4d2b-9362-fbeffa057c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:45.784244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 675/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:09:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:47.656055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39cc36-b44e-4536-96b5-fe83154d85a7 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:47.656596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39cc36-b44e-4536-96b5-fe83154d85a7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:47.656741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39cc36-b44e-4536-96b5-fe83154d85a7 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:47.658568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 679/1354] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:47.663728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91e4a2-a41d-4763-846a-7457e36bddd5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:47.664557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91e4a2-a41d-4763-846a-7457e36bddd5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:47.664848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91e4a2-a41d-4763-846a-7457e36bddd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:47.684169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ed91e4a2-a41d-4763-846a-7457e36bddd5 None None] MFA Rules not processed for user `7ee40bcb7905490c9ad3e39e7a93b122`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:47.685480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ed91e4a2-a41d-4763-846a-7457e36bddd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:47.695101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 676/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:09:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:51.605612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf326e-075e-4c0d-a2b2-8eece6ea6678 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:51.605612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf326e-075e-4c0d-a2b2-8eece6ea6678 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:51.606563 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf326e-075e-4c0d-a2b2-8eece6ea6678 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:51.613129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0eeaf1-5551-41bd-aa0c-7664d381b93d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:51.614027 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0eeaf1-5551-41bd-aa0c-7664d381b93d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:51.614456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0eeaf1-5551-41bd-aa0c-7664d381b93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:51.616775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 680/1356] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:51 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:51.637945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-cc0eeaf1-5551-41bd-aa0c-7664d381b93d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:51.639338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cc0eeaf1-5551-41bd-aa0c-7664d381b93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:51.654327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 677/1357] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:51.715904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233b283-fb3d-4280-a1c0-2e7e6dd34d2c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:51.716677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233b283-fb3d-4280-a1c0-2e7e6dd34d2c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:51.717584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233b283-fb3d-4280-a1c0-2e7e6dd34d2c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:51.720516 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 681/1358] 10.222.0.46 () {54 vars in 961 bytes} [Thu May 26 18:09:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:51.733454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50d887-c227-41d5-bc32-c82cd038c2af None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:51.734237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50d887-c227-41d5-bc32-c82cd038c2af None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:51.738637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50d887-c227-41d5-bc32-c82cd038c2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:51.797162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:51.808280 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-fe50d887-c227-41d5-bc32-c82cd038c2af None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:51.809401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-fe50d887-c227-41d5-bc32-c82cd038c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:51.910035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 678/1359] 10.222.0.46 () {58 vars in 1058 bytes} [Thu May 26 18:09:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:51.918586 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233b283-fb3d-4280-a1c0-2e7e6dd34d2c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:51.933561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233b283-fb3d-4280-a1c0-2e7e6dd34d2c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:51.939694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68da4b-c3e9-4295-9e3d-e20dd8c9cda7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:51.940372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68da4b-c3e9-4295-9e3d-e20dd8c9cda7 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:51.940669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68da4b-c3e9-4295-9e3d-e20dd8c9cda7 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:51.940969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68da4b-c3e9-4295-9e3d-e20dd8c9cda7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:51.951448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68da4b-c3e9-4295-9e3d-e20dd8c9cda7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:51.953221 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68da4b-c3e9-4295-9e3d-e20dd8c9cda7 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:51.964423 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 682/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:54.682731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f848695-5ab3-47f0-891e-e9094a83b816 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:54.683332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f848695-5ab3-47f0-891e-e9094a83b816 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:54.683332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f848695-5ab3-47f0-891e-e9094a83b816 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:54.685182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 679/1361] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:54.690162 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0bfa8-02c8-4ced-b9e5-1430452ef1b9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:54.691000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0bfa8-02c8-4ced-b9e5-1430452ef1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:54.691276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0bfa8-02c8-4ced-b9e5-1430452ef1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:54.717287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f9b0bfa8-02c8-4ced-b9e5-1430452ef1b9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:54.718520 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f9b0bfa8-02c8-4ced-b9e5-1430452ef1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:54.729553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 683/1362] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:54.794711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f848695-5ab3-47f0-891e-e9094a83b816 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:54.822095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f848695-5ab3-47f0-891e-e9094a83b816 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:54.830454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa87a2-a6fc-4540-bb9d-0f233fc02c7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:54.831477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa87a2-a6fc-4540-bb9d-0f233fc02c7b service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:54.831938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa87a2-a6fc-4540-bb9d-0f233fc02c7b service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:54.832392 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa87a2-a6fc-4540-bb9d-0f233fc02c7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:54.847639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa87a2-a6fc-4540-bb9d-0f233fc02c7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:54.849836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa87a2-a6fc-4540-bb9d-0f233fc02c7b service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:54.864195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 680/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:57.571662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ae1f-6dae-4843-84d6-04228e2f4dd5 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:57.572170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ae1f-6dae-4843-84d6-04228e2f4dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:57.572283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ae1f-6dae-4843-84d6-04228e2f4dd5 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:57.573687 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 684/1364] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:57.579517 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f65a9-f564-4cd8-b20b-1f135e4e1554 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:57.580022 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f65a9-f564-4cd8-b20b-1f135e4e1554 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:57.580111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f65a9-f564-4cd8-b20b-1f135e4e1554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:57.598840 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-159f65a9-f564-4cd8-b20b-1f135e4e1554 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:57.600607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-159f65a9-f564-4cd8-b20b-1f135e4e1554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:57.608782 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 681/1365] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:57.655185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73184b18-706a-435f-9930-983692d78039 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:57.655600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73184b18-706a-435f-9930-983692d78039 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:57.656086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73184b18-706a-435f-9930-983692d78039 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:57.657639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 685/1366] 10.222.0.46 () {54 vars in 961 bytes} [Thu May 26 18:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:09:57.666594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a1979-d103-47e2-b12d-76648004e7a4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:57.666594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a1979-d103-47e2-b12d-76648004e7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:57.666594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a1979-d103-47e2-b12d-76648004e7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:57.685956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-518a1979-d103-47e2-b12d-76648004e7a4 None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:57.687213 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-518a1979-d103-47e2-b12d-76648004e7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:57.697501 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 682/1367] 10.222.0.46 () {58 vars in 1058 bytes} [Thu May 26 18:09:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:57.705608 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73184b18-706a-435f-9930-983692d78039 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:57.732557 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73184b18-706a-435f-9930-983692d78039 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:57.738893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acddc50-5d5c-4095-a4d2-17a43cc60700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:57.739555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acddc50-5d5c-4095-a4d2-17a43cc60700 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:57.740496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acddc50-5d5c-4095-a4d2-17a43cc60700 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:57.740832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acddc50-5d5c-4095-a4d2-17a43cc60700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:57.749779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acddc50-5d5c-4095-a4d2-17a43cc60700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:57.751409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acddc50-5d5c-4095-a4d2-17a43cc60700 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:57.763902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 686/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:00.216856 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55974924-eaae-4774-b9a9-e243257e8dff None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:00.217393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55974924-eaae-4774-b9a9-e243257e8dff None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:00.217393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55974924-eaae-4774-b9a9-e243257e8dff None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:00.218806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 683/1369] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:00.224915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4555ebb-ae62-41a7-838d-e5b7b3815b29 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:00.225382 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4555ebb-ae62-41a7-838d-e5b7b3815b29 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:00.225674 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4555ebb-ae62-41a7-838d-e5b7b3815b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:00.245537 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e4555ebb-ae62-41a7-838d-e5b7b3815b29 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:00.246775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e4555ebb-ae62-41a7-838d-e5b7b3815b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:00.255598 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 687/1370] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:00.304667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55974924-eaae-4774-b9a9-e243257e8dff None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:00.323741 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55974924-eaae-4774-b9a9-e243257e8dff None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:00.330012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fc37d-ab0d-430c-a6e3-34c80a93e254 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:00.330688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fc37d-ab0d-430c-a6e3-34c80a93e254 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:00.330974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fc37d-ab0d-430c-a6e3-34c80a93e254 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:00.331252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fc37d-ab0d-430c-a6e3-34c80a93e254 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:00.340933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fc37d-ab0d-430c-a6e3-34c80a93e254 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:00.342576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fc37d-ab0d-430c-a6e3-34c80a93e254 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:00.365559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 684/1371] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:00.652689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a843bf-db3e-4c58-8dbf-a31fdcd851bc None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:00.652979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a843bf-db3e-4c58-8dbf-a31fdcd851bc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:00.653428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a843bf-db3e-4c58-8dbf-a31fdcd851bc None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:00.654859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 688/1372] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:00.660622 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85476eb-f4fb-428e-8dde-8e1cb7979ffc None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:00.660905 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85476eb-f4fb-428e-8dde-8e1cb7979ffc None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:00.661362 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85476eb-f4fb-428e-8dde-8e1cb7979ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:00.680789 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e85476eb-f4fb-428e-8dde-8e1cb7979ffc None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:00.682095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e85476eb-f4fb-428e-8dde-8e1cb7979ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:00.690790 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 685/1373] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:10:00.971676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e760f98-349f-4797-ad30-016d4fb028a8 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:00.971676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e760f98-349f-4797-ad30-016d4fb028a8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:00.971676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e760f98-349f-4797-ad30-016d4fb028a8 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:00.971676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 689/1374] 10.222.0.46 () {54 vars in 963 bytes} [Thu May 26 18:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:00.978731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026a4d8-f282-4e04-95ae-a97d6e7a5076 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:00.979337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026a4d8-f282-4e04-95ae-a97d6e7a5076 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:00.979962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026a4d8-f282-4e04-95ae-a97d6e7a5076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:01.059335 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-8026a4d8-f282-4e04-95ae-a97d6e7a5076 None None] MFA Rules not processed for user `8c79367697f7451fb804b8641444f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:01.060817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8026a4d8-f282-4e04-95ae-a97d6e7a5076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:01.143601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 686/1375] 10.222.0.46 () {58 vars in 1060 bytes} [Thu May 26 18:10:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:01.152633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e760f98-349f-4797-ad30-016d4fb028a8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:01.165887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e760f98-349f-4797-ad30-016d4fb028a8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:01.170731 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f214eee3-17a6-4f64-9b95-40748280eaae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:01.171532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214eee3-17a6-4f64-9b95-40748280eaae service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:01.171925 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214eee3-17a6-4f64-9b95-40748280eaae service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:01.172465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214eee3-17a6-4f64-9b95-40748280eaae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:01.184257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f214eee3-17a6-4f64-9b95-40748280eaae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:01.185711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f214eee3-17a6-4f64-9b95-40748280eaae service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:01.201795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 690/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:02.751236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6df6f6-a277-4df2-89af-ca26dad6c2d0 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:02.751807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6df6f6-a277-4df2-89af-ca26dad6c2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:02.751807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6df6f6-a277-4df2-89af-ca26dad6c2d0 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:02.753219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 687/1377] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:02.759546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe1134-cad2-49f3-a1f0-138f2a488f10 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:02.760281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe1134-cad2-49f3-a1f0-138f2a488f10 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:02.760281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe1134-cad2-49f3-a1f0-138f2a488f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:02.779782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c5fe1134-cad2-49f3-a1f0-138f2a488f10 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:02.781026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c5fe1134-cad2-49f3-a1f0-138f2a488f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:02.789989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 691/1378] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:02.838378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6df6f6-a277-4df2-89af-ca26dad6c2d0 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:02.855130 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6df6f6-a277-4df2-89af-ca26dad6c2d0 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:02.860805 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b6bb5-ad10-4165-b44d-5210af9d35a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:02.861474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b6bb5-ad10-4165-b44d-5210af9d35a5 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:02.861791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b6bb5-ad10-4165-b44d-5210af9d35a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:02.862325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b6bb5-ad10-4165-b44d-5210af9d35a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:02.872438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b6bb5-ad10-4165-b44d-5210af9d35a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:02.874138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b6bb5-ad10-4165-b44d-5210af9d35a5 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:02.878865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:02.887360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 688/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:05.272115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d555a35-2154-471d-acf8-67fc122aa38d None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:05.273354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d555a35-2154-471d-acf8-67fc122aa38d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:05.274049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d555a35-2154-471d-acf8-67fc122aa38d None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:05.277100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 692/1380] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:10:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:05.282752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d113be-e21e-4562-95ef-1a05f50a0580 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:05.283372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d113be-e21e-4562-95ef-1a05f50a0580 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:05.283372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d113be-e21e-4562-95ef-1a05f50a0580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:05.304394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-21d113be-e21e-4562-95ef-1a05f50a0580 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:05.305658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-21d113be-e21e-4562-95ef-1a05f50a0580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:05.323050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 689/1381] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:10:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:05.370954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d555a35-2154-471d-acf8-67fc122aa38d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:05.387438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d555a35-2154-471d-acf8-67fc122aa38d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:05.393220 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f65513-3d1b-4544-838a-a79a3c3260c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:05.393900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f65513-3d1b-4544-838a-a79a3c3260c8 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:05.394184 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f65513-3d1b-4544-838a-a79a3c3260c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:05.394457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f65513-3d1b-4544-838a-a79a3c3260c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:05.403804 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f65513-3d1b-4544-838a-a79a3c3260c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:05.405461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f65513-3d1b-4544-838a-a79a3c3260c8 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:05.417299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 693/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:07.795553 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373c9ed-1c65-4738-9600-9935a8c6f5c4 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:07.796100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373c9ed-1c65-4738-9600-9935a8c6f5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:07.796100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373c9ed-1c65-4738-9600-9935a8c6f5c4 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:07.797799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 690/1383] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:07.807477 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776af49b-07b2-4556-a559-ad6b0ea1c177 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:07.808159 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776af49b-07b2-4556-a559-ad6b0ea1c177 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:07.808739 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776af49b-07b2-4556-a559-ad6b0ea1c177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:07.830038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-776af49b-07b2-4556-a559-ad6b0ea1c177 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:07.831267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-776af49b-07b2-4556-a559-ad6b0ea1c177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:07.839949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 694/1384] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:10:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:07.885371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373c9ed-1c65-4738-9600-9935a8c6f5c4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:07.901140 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373c9ed-1c65-4738-9600-9935a8c6f5c4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:07.906843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da81b5-5b9c-4803-8a5e-862b6556807a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:07.907589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da81b5-5b9c-4803-8a5e-862b6556807a service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:07.907948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da81b5-5b9c-4803-8a5e-862b6556807a service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:07.908296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da81b5-5b9c-4803-8a5e-862b6556807a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:07.918539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da81b5-5b9c-4803-8a5e-862b6556807a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:07.920273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da81b5-5b9c-4803-8a5e-862b6556807a service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:07.933367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 691/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:10.356696 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65e9bf-9b7b-4eee-95aa-31216bb02cf5 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:10.357278 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65e9bf-9b7b-4eee-95aa-31216bb02cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:10.357278 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65e9bf-9b7b-4eee-95aa-31216bb02cf5 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:10.358741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 695/1386] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:10.365672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc80338-f10a-4f8f-9886-daee3ae7c129 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:10.366865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc80338-f10a-4f8f-9886-daee3ae7c129 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:10.366865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc80338-f10a-4f8f-9886-daee3ae7c129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:10.391115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0dc80338-f10a-4f8f-9886-daee3ae7c129 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:10.392380 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0dc80338-f10a-4f8f-9886-daee3ae7c129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:10.400965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 692/1387] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:10.449042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65e9bf-9b7b-4eee-95aa-31216bb02cf5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:10.470579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65e9bf-9b7b-4eee-95aa-31216bb02cf5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:10.488344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93d525-a46c-4607-9245-c8dcf05bc1a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:10.490130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93d525-a46c-4607-9245-c8dcf05bc1a6 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:10.490603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93d525-a46c-4607-9245-c8dcf05bc1a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:10.491037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93d525-a46c-4607-9245-c8dcf05bc1a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:10.505817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93d525-a46c-4607-9245-c8dcf05bc1a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:10.508618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93d525-a46c-4607-9245-c8dcf05bc1a6 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:10.521607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 696/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:13.186832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fe13c-e146-4891-ad2f-c2cd2428bfe5 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:13.187631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fe13c-e146-4891-ad2f-c2cd2428bfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:13.187631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fe13c-e146-4891-ad2f-c2cd2428bfe5 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:13.188846 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 693/1389] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:13.196079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806bd9e-06e7-4cba-b682-403539479e55 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:13.198424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806bd9e-06e7-4cba-b682-403539479e55 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:13.199360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806bd9e-06e7-4cba-b682-403539479e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:13.224045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9806bd9e-06e7-4cba-b682-403539479e55 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:13.225212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9806bd9e-06e7-4cba-b682-403539479e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:13.233788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 697/1390] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:10:13.287947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fe13c-e146-4891-ad2f-c2cd2428bfe5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:13.304286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fe13c-e146-4891-ad2f-c2cd2428bfe5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:13.310214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77861c-b525-4fdc-bdcc-1d16ca5073a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:13.310870 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77861c-b525-4fdc-bdcc-1d16ca5073a3 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:13.311149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77861c-b525-4fdc-bdcc-1d16ca5073a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:13.311422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77861c-b525-4fdc-bdcc-1d16ca5073a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:13.326251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77861c-b525-4fdc-bdcc-1d16ca5073a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:13.330014 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77861c-b525-4fdc-bdcc-1d16ca5073a3 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:13.354523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 694/1391] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:10:16.886678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e57a1-6a13-4b2e-8b39-14d9cf5a3106 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:16.887296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e57a1-6a13-4b2e-8b39-14d9cf5a3106 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:16.887296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e57a1-6a13-4b2e-8b39-14d9cf5a3106 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:16.888621 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 698/1392] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:16.894452 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c6042-f4b4-41f3-86ed-0387dc147727 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:16.894907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c6042-f4b4-41f3-86ed-0387dc147727 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:16.895135 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c6042-f4b4-41f3-86ed-0387dc147727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:16.917382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e67c6042-f4b4-41f3-86ed-0387dc147727 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:16.918619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e67c6042-f4b4-41f3-86ed-0387dc147727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:16.929906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 695/1393] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:16.979661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e57a1-6a13-4b2e-8b39-14d9cf5a3106 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:16.987924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=46 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:16.996094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e57a1-6a13-4b2e-8b39-14d9cf5a3106 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:17.002710 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bcf40-fdad-48b6-8b58-79c42eb66446 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:17.003372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bcf40-fdad-48b6-8b58-79c42eb66446 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:17.003650 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bcf40-fdad-48b6-8b58-79c42eb66446 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:17.003923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bcf40-fdad-48b6-8b58-79c42eb66446 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:17.013832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55bcf40-fdad-48b6-8b58-79c42eb66446 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:17.015828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55bcf40-fdad-48b6-8b58-79c42eb66446 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:17.026845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 699/1394] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:10:17.624408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60349465-844c-438c-9727-988cba42fa33 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:17.624621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60349465-844c-438c-9727-988cba42fa33 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:17.624866 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60349465-844c-438c-9727-988cba42fa33 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:17.626465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 696/1395] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:17.633121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1a68b-7a13-4feb-aac7-9897ff612dd2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:17.633391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1a68b-7a13-4feb-aac7-9897ff612dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:17.633666 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1a68b-7a13-4feb-aac7-9897ff612dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:17.653893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d7e1a68b-7a13-4feb-aac7-9897ff612dd2 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:17.654402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d7e1a68b-7a13-4feb-aac7-9897ff612dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:17.663191 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 700/1396] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:10:17.720681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430279d2-c239-4427-bd89-26696b5b7680 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:17.720976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430279d2-c239-4427-bd89-26696b5b7680 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:17.721254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430279d2-c239-4427-bd89-26696b5b7680 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:17.722847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 697/1397] 10.222.0.46 () {54 vars in 962 bytes} [Thu May 26 18:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:17.728441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fd50f-78a9-4f92-af6b-cc11e3483b97 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:17.728753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fd50f-78a9-4f92-af6b-cc11e3483b97 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:17.729004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fd50f-78a9-4f92-af6b-cc11e3483b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:17.784789 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-605fd50f-78a9-4f92-af6b-cc11e3483b97 None None] MFA Rules not processed for user `4a67120485f74f8bb0ff188cd3abd21c`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:17.786277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-605fd50f-78a9-4f92-af6b-cc11e3483b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:17.851637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 701/1398] 10.222.0.46 () {58 vars in 1059 bytes} [Thu May 26 18:10:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:17.866963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430279d2-c239-4427-bd89-26696b5b7680 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:17.875876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430279d2-c239-4427-bd89-26696b5b7680 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:17.880303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7592a22-794a-4dd8-8756-3d08f706dedc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a67120485f74f8bb0ff188cd3abd21c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:17.881080 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7592a22-794a-4dd8-8756-3d08f706dedc service glance] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:17.881473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7592a22-794a-4dd8-8756-3d08f706dedc service glance] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:17.881896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7592a22-794a-4dd8-8756-3d08f706dedc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:17.892190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7592a22-794a-4dd8-8756-3d08f706dedc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:17.893638 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7592a22-794a-4dd8-8756-3d08f706dedc service glance] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:17.904989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 698/1399] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:10:18.478105 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24b593-5978-41c1-bd4e-fbe2ce69fb8a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:18.478612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24b593-5978-41c1-bd4e-fbe2ce69fb8a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:18.478709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24b593-5978-41c1-bd4e-fbe2ce69fb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:18.537409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ce24b593-5978-41c1-bd4e-fbe2ce69fb8a None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:18.538774 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ce24b593-5978-41c1-bd4e-fbe2ce69fb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:18.617142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 702/1400] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:10:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:18.661288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830d656-d3f2-4e7a-a4e5-c800311771d1 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:18.661669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830d656-d3f2-4e7a-a4e5-c800311771d1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:18.661982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830d656-d3f2-4e7a-a4e5-c800311771d1 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:18.664240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 699/1401] 10.222.0.46 () {54 vars in 942 bytes} [Thu May 26 18:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:18.671240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aae273-55ce-46e9-97b2-1fc44f719d48 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:18.671605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aae273-55ce-46e9-97b2-1fc44f719d48 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:18.671952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aae273-55ce-46e9-97b2-1fc44f719d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:18.727856 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:18.731557 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-26aae273-55ce-46e9-97b2-1fc44f719d48 None None] MFA Rules not processed for user `d656b42c2acb498caa92c77a22fd7efe`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:18.733632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-26aae273-55ce-46e9-97b2-1fc44f719d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:18.814223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 703/1402] 10.222.0.46 () {58 vars in 1039 bytes} [Thu May 26 18:10:18 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:10:18.825304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830d656-d3f2-4e7a-a4e5-c800311771d1 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:18.846193 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830d656-d3f2-4e7a-a4e5-c800311771d1 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:18.854134 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d184bb-a7ad-44b6-a3ff-72eb5127ed74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:18.855104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d184bb-a7ad-44b6-a3ff-72eb5127ed74 service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:18.855526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d184bb-a7ad-44b6-a3ff-72eb5127ed74 service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:18.855959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d184bb-a7ad-44b6-a3ff-72eb5127ed74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:18.867413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d184bb-a7ad-44b6-a3ff-72eb5127ed74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:18.869059 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d184bb-a7ad-44b6-a3ff-72eb5127ed74 service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:18.882619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 700/1403] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:10:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:10:19.442983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95120d-e117-4a27-bdfc-00b57a47d522 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:19.443295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95120d-e117-4a27-bdfc-00b57a47d522 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:19.443537 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95120d-e117-4a27-bdfc-00b57a47d522 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:19.445055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 704/1404] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:19.450825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dd122-dd5a-4577-bd4a-025cb849ac7e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:19.451535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dd122-dd5a-4577-bd4a-025cb849ac7e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:19.451535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dd122-dd5a-4577-bd4a-025cb849ac7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:19.470439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-678dd122-dd5a-4577-bd4a-025cb849ac7e None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:19.472054 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-678dd122-dd5a-4577-bd4a-025cb849ac7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:19.480992 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 701/1405] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:19.527552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95120d-e117-4a27-bdfc-00b57a47d522 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:19.543513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95120d-e117-4a27-bdfc-00b57a47d522 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:19.549369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127bb75-7eca-4267-9aa8-601e7b9ee8a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:19.550055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127bb75-7eca-4267-9aa8-601e7b9ee8a7 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:19.550535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127bb75-7eca-4267-9aa8-601e7b9ee8a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:19.550819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127bb75-7eca-4267-9aa8-601e7b9ee8a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:19.562852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127bb75-7eca-4267-9aa8-601e7b9ee8a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:19.564520 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127bb75-7eca-4267-9aa8-601e7b9ee8a7 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:19.579028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 705/1406] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:22.002754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20577073-b3ff-4f1e-953d-20f69b70cf0b None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:22.003551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20577073-b3ff-4f1e-953d-20f69b70cf0b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:22.003551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20577073-b3ff-4f1e-953d-20f69b70cf0b None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:22.004962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 702/1407] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:22.010794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5258db-bb39-47f3-92a7-ab74c4c5a0d3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:22.011639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5258db-bb39-47f3-92a7-ab74c4c5a0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:22.012135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5258db-bb39-47f3-92a7-ab74c4c5a0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:22.043958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ee5258db-bb39-47f3-92a7-ab74c4c5a0d3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:22.046436 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ee5258db-bb39-47f3-92a7-ab74c4c5a0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:22.058969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 706/1408] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:22.111137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20577073-b3ff-4f1e-953d-20f69b70cf0b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:22.127223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20577073-b3ff-4f1e-953d-20f69b70cf0b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:22.133819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dc6a1-c01a-4107-9d5e-e8c697907b42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:22.134801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dc6a1-c01a-4107-9d5e-e8c697907b42 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:22.135149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dc6a1-c01a-4107-9d5e-e8c697907b42 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:22.135476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dc6a1-c01a-4107-9d5e-e8c697907b42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:22.146115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dc6a1-c01a-4107-9d5e-e8c697907b42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:22.148628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dc6a1-c01a-4107-9d5e-e8c697907b42 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:22.159496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 703/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:10:32.033839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=39 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:32.154607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=43 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:43.201214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410976f-6459-4bc3-ac8a-c629b800c24c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:43.201847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410976f-6459-4bc3-ac8a-c629b800c24c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:43.201847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410976f-6459-4bc3-ac8a-c629b800c24c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:43.203255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 707/1410] 10.222.0.46 () {54 vars in 918 bytes} [Thu May 26 18:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:10:43.208901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d26af3-cc70-4026-a786-330aec43f876 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:43.209386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d26af3-cc70-4026-a786-330aec43f876 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:43.209498 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d26af3-cc70-4026-a786-330aec43f876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:43.227752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b5d26af3-cc70-4026-a786-330aec43f876 None None] MFA Rules not processed for user `7ee40bcb7905490c9ad3e39e7a93b122`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:43.229964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b5d26af3-cc70-4026-a786-330aec43f876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:43.239689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 704/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Thu May 26 18:10:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:10:43.255728 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cd182-ceac-4676-a2e9-e98ac8ae82f8 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:43.256016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cd182-ceac-4676-a2e9-e98ac8ae82f8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:43.256290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cd182-ceac-4676-a2e9-e98ac8ae82f8 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:43.257718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 708/1412] 10.222.0.46 () {54 vars in 918 bytes} [Thu May 26 18:10:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:43.263259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575b8b9-54e6-4714-959b-823200d873fd None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:43.263632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575b8b9-54e6-4714-959b-823200d873fd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:43.264000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575b8b9-54e6-4714-959b-823200d873fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:43.285491 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9575b8b9-54e6-4714-959b-823200d873fd None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:43.286656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9575b8b9-54e6-4714-959b-823200d873fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:43.296334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 705/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Thu May 26 18:10:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:46.042020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4947a04-bf29-48e1-a982-c28866ab802a None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:46.042602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4947a04-bf29-48e1-a982-c28866ab802a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:46.042602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4947a04-bf29-48e1-a982-c28866ab802a None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:46.043943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:46.049318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f4315-78fd-44a0-af55-2d716c06f801 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:46.049800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f4315-78fd-44a0-af55-2d716c06f801 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:46.049907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f4315-78fd-44a0-af55-2d716c06f801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:46.067887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-028f4315-78fd-44a0-af55-2d716c06f801 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:46.069087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-028f4315-78fd-44a0-af55-2d716c06f801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:46.076800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:46.120080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4947a04-bf29-48e1-a982-c28866ab802a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:46.142870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4947a04-bf29-48e1-a982-c28866ab802a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:46.151875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613814d-26e3-4cb9-9521-9b9141631ab3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:46.152935 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613814d-26e3-4cb9-9521-9b9141631ab3 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:46.153426 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613814d-26e3-4cb9-9521-9b9141631ab3 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:46.153977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613814d-26e3-4cb9-9521-9b9141631ab3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:46.167914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613814d-26e3-4cb9-9521-9b9141631ab3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:46.169534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613814d-26e3-4cb9-9521-9b9141631ab3 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:46.179102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 710/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:47.178580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad90691-460e-4c6b-ac0e-9445049ca89d None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:47.179151 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad90691-460e-4c6b-ac0e-9445049ca89d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:47.179151 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad90691-460e-4c6b-ac0e-9445049ca89d None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:47.180509 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:47.185632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fa458-e911-4926-b672-df89ebcc9174 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:47.186054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fa458-e911-4926-b672-df89ebcc9174 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:47.186221 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fa458-e911-4926-b672-df89ebcc9174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:47.203914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-564fa458-e911-4926-b672-df89ebcc9174 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:47.205117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-564fa458-e911-4926-b672-df89ebcc9174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:47.212906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:47.369721 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad90691-460e-4c6b-ac0e-9445049ca89d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:47.383864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad90691-460e-4c6b-ac0e-9445049ca89d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:47.389153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065fcda-fa38-4823-bfa2-487f3c5dab74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:47.389824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065fcda-fa38-4823-bfa2-487f3c5dab74 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:47.390109 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065fcda-fa38-4823-bfa2-487f3c5dab74 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:47.390382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065fcda-fa38-4823-bfa2-487f3c5dab74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:47.398753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065fcda-fa38-4823-bfa2-487f3c5dab74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:47.400346 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065fcda-fa38-4823-bfa2-487f3c5dab74 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:47.410588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 708/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:48.349428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b1f04-fa40-4b32-8547-5e7a04df7c63 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.349972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b1f04-fa40-4b32-8547-5e7a04df7c63 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.349972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b1f04-fa40-4b32-8547-5e7a04df7c63 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.351372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:10:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:48.356442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e98b72-b101-41fc-8eb8-93523558fbfe None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.356857 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e98b72-b101-41fc-8eb8-93523558fbfe None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.357105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e98b72-b101-41fc-8eb8-93523558fbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.375003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-87e98b72-b101-41fc-8eb8-93523558fbfe None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:48.376179 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-87e98b72-b101-41fc-8eb8-93523558fbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:48.383825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:10:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:48.538765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b1f04-fa40-4b32-8547-5e7a04df7c63 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:48.553162 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b1f04-fa40-4b32-8547-5e7a04df7c63 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:48.558594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb43b32-48e3-4546-827c-bd5071102c1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:48.559229 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb43b32-48e3-4546-827c-bd5071102c1d service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.559507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb43b32-48e3-4546-827c-bd5071102c1d service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.559776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb43b32-48e3-4546-827c-bd5071102c1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.568079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb43b32-48e3-4546-827c-bd5071102c1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:48.569750 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb43b32-48e3-4546-827c-bd5071102c1d service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:48.579108 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 713/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:49.505376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8981c-ed4c-4c13-a776-40553bab4743 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:49.505971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8981c-ed4c-4c13-a776-40553bab4743 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:49.505971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8981c-ed4c-4c13-a776-40553bab4743 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:49.507474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:10:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:49.512977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70577dc2-c03b-4957-9e80-7c22d92191a2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:49.513527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70577dc2-c03b-4957-9e80-7c22d92191a2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:49.513527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70577dc2-c03b-4957-9e80-7c22d92191a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:49.532345 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-70577dc2-c03b-4957-9e80-7c22d92191a2 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:49.533543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-70577dc2-c03b-4957-9e80-7c22d92191a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:49.541977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:49.694323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8981c-ed4c-4c13-a776-40553bab4743 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:49.709012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8981c-ed4c-4c13-a776-40553bab4743 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:49.714602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb0b25-b548-402d-9ddc-bc11f2cf675d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:49.715237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb0b25-b548-402d-9ddc-bc11f2cf675d service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:49.715518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb0b25-b548-402d-9ddc-bc11f2cf675d service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:49.715788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb0b25-b548-402d-9ddc-bc11f2cf675d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:49.724078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb0b25-b548-402d-9ddc-bc11f2cf675d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:49.725642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb0b25-b548-402d-9ddc-bc11f2cf675d service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:49.734904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 711/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:50.657511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3472e-40dc-41b3-9a8d-44b24fdac711 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:50.658168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3472e-40dc-41b3-9a8d-44b24fdac711 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:50.658168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3472e-40dc-41b3-9a8d-44b24fdac711 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:50.659499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:50.664783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406dc0b-177e-46d7-8323-67387158269d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:50.665198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406dc0b-177e-46d7-8323-67387158269d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:50.665319 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406dc0b-177e-46d7-8323-67387158269d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:50.683755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9406dc0b-177e-46d7-8323-67387158269d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:50.684952 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9406dc0b-177e-46d7-8323-67387158269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:50.693349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:50.841319 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3472e-40dc-41b3-9a8d-44b24fdac711 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:50.859227 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3472e-40dc-41b3-9a8d-44b24fdac711 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:50.865071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98902f6a-bdbc-4436-92f4-abf0bc08c2e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:50.865702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98902f6a-bdbc-4436-92f4-abf0bc08c2e3 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:50.866014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98902f6a-bdbc-4436-92f4-abf0bc08c2e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:50.866284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98902f6a-bdbc-4436-92f4-abf0bc08c2e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:50.874487 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98902f6a-bdbc-4436-92f4-abf0bc08c2e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:50.876091 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98902f6a-bdbc-4436-92f4-abf0bc08c2e3 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:50.886080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 716/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:51.743384 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b3b97-fa18-4dfc-b0da-d1e8b59cc4e2 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:51.743923 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b3b97-fa18-4dfc-b0da-d1e8b59cc4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:51.743923 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b3b97-fa18-4dfc-b0da-d1e8b59cc4e2 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:51.745314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:51.750723 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c090b-1b3f-4a58-b166-e4995a631530 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:51.751144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c090b-1b3f-4a58-b166-e4995a631530 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:51.751270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c090b-1b3f-4a58-b166-e4995a631530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:51.770136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6a4c090b-1b3f-4a58-b166-e4995a631530 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:51.771407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6a4c090b-1b3f-4a58-b166-e4995a631530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:51.781096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:51.937335 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b3b97-fa18-4dfc-b0da-d1e8b59cc4e2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:51.951618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b3b97-fa18-4dfc-b0da-d1e8b59cc4e2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:51.956960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9127717-1a0c-4c80-9c68-b07335379b05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:51.957822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9127717-1a0c-4c80-9c68-b07335379b05 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:51.958112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9127717-1a0c-4c80-9c68-b07335379b05 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:51.958388 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9127717-1a0c-4c80-9c68-b07335379b05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:51.966483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9127717-1a0c-4c80-9c68-b07335379b05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:51.968084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9127717-1a0c-4c80-9c68-b07335379b05 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:51.977952 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 714/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:52.846780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d7c55-f90c-47d6-a3af-68be5f0b9723 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:52.847968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d7c55-f90c-47d6-a3af-68be5f0b9723 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:52.847968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d7c55-f90c-47d6-a3af-68be5f0b9723 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:52.849395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:10:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:52.854567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d51e60-fe0b-4993-ab22-3b5b2f6a67d3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:52.854986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d51e60-fe0b-4993-ab22-3b5b2f6a67d3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:52.855123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d51e60-fe0b-4993-ab22-3b5b2f6a67d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:52.874819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-54d51e60-fe0b-4993-ab22-3b5b2f6a67d3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:52.876026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-54d51e60-fe0b-4993-ab22-3b5b2f6a67d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:52.885476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:53.039186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d7c55-f90c-47d6-a3af-68be5f0b9723 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:53.054310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d7c55-f90c-47d6-a3af-68be5f0b9723 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:53.059592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b82074-56f7-4d2b-8a02-6b3198c40e86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:53.060221 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b82074-56f7-4d2b-8a02-6b3198c40e86 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:53.060503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b82074-56f7-4d2b-8a02-6b3198c40e86 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:53.060776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b82074-56f7-4d2b-8a02-6b3198c40e86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:53.069812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b82074-56f7-4d2b-8a02-6b3198c40e86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:53.071428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b82074-56f7-4d2b-8a02-6b3198c40e86 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:53.080962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 719/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:53.945321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413573b-0c88-4fc6-8382-1a7feb114670 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:53.945899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413573b-0c88-4fc6-8382-1a7feb114670 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:53.945899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413573b-0c88-4fc6-8382-1a7feb114670 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:53.947324 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:53.953032 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6320c-76af-4efb-a3b1-d71e43671fc8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:53.953877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6320c-76af-4efb-a3b1-d71e43671fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:53.953877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6320c-76af-4efb-a3b1-d71e43671fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:53.972672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-18b6320c-76af-4efb-a3b1-d71e43671fc8 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:53.973914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-18b6320c-76af-4efb-a3b1-d71e43671fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:53.981612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:10:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:10:54.135041 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413573b-0c88-4fc6-8382-1a7feb114670 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:54.150007 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413573b-0c88-4fc6-8382-1a7feb114670 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:54.155309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf07be-1051-49ed-8c03-90e6470b7eb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:54.155933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf07be-1051-49ed-8c03-90e6470b7eb0 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:54.156211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf07be-1051-49ed-8c03-90e6470b7eb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:54.156480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf07be-1051-49ed-8c03-90e6470b7eb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:54.165018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf07be-1051-49ed-8c03-90e6470b7eb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:54.166863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf07be-1051-49ed-8c03-90e6470b7eb0 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:54.176809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 717/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:55.061857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b0032-468b-4211-a88f-51deb16fb920 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:55.062407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b0032-468b-4211-a88f-51deb16fb920 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:55.062407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b0032-468b-4211-a88f-51deb16fb920 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:55.063773 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:55.069683 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbdd9e-5950-4484-965e-e316ea02e6b6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:55.070148 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbdd9e-5950-4484-965e-e316ea02e6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:55.070252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbdd9e-5950-4484-965e-e316ea02e6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:55.088005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-cccbdd9e-5950-4484-965e-e316ea02e6b6 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:55.089187 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cccbdd9e-5950-4484-965e-e316ea02e6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:55.096906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:10:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:55.245346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b0032-468b-4211-a88f-51deb16fb920 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:55.260186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b0032-468b-4211-a88f-51deb16fb920 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:55.265513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebb42a-2580-4bb8-bd75-5162c7eb1983 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:55.266178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebb42a-2580-4bb8-bd75-5162c7eb1983 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:55.266460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebb42a-2580-4bb8-bd75-5162c7eb1983 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:55.266732 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebb42a-2580-4bb8-bd75-5162c7eb1983 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:55.274910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebb42a-2580-4bb8-bd75-5162c7eb1983 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:55.276504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebb42a-2580-4bb8-bd75-5162c7eb1983 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:55.286282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 722/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:57.451869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b33e70-2165-46d9-bfa1-051300aed93f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:57.452435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b33e70-2165-46d9-bfa1-051300aed93f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:57.452435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b33e70-2165-46d9-bfa1-051300aed93f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:57.453854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:10:57.460486 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a69ec-c36e-46ed-bed4-0ef3a5cabe70 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:57.461167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a69ec-c36e-46ed-bed4-0ef3a5cabe70 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:57.461167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a69ec-c36e-46ed-bed4-0ef3a5cabe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:57.480880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6c5a69ec-c36e-46ed-bed4-0ef3a5cabe70 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:57.482153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6c5a69ec-c36e-46ed-bed4-0ef3a5cabe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:57.490972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:57.529485 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b33e70-2165-46d9-bfa1-051300aed93f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:57.542673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b33e70-2165-46d9-bfa1-051300aed93f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:57.548298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923b326-faa6-4428-ac2e-bd53f6085b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:57.548953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923b326-faa6-4428-ac2e-bd53f6085b35 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:57.549242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923b326-faa6-4428-ac2e-bd53f6085b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:57.549518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923b326-faa6-4428-ac2e-bd53f6085b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:57.560042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923b326-faa6-4428-ac2e-bd53f6085b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:57.561538 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923b326-faa6-4428-ac2e-bd53f6085b35 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:57.572490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 720/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:10:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:00.187758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42469b8c-6ce2-41dd-b005-71360286a5ee None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:00.188301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42469b8c-6ce2-41dd-b005-71360286a5ee None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:00.188301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42469b8c-6ce2-41dd-b005-71360286a5ee None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:00.189651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:00.195879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ff2ef-c83c-4f81-a9d6-83922a919528 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:00.196320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ff2ef-c83c-4f81-a9d6-83922a919528 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:00.196415 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ff2ef-c83c-4f81-a9d6-83922a919528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:00.214336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ea1ff2ef-c83c-4f81-a9d6-83922a919528 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:00.215516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ea1ff2ef-c83c-4f81-a9d6-83922a919528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:00.223412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:00.262705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42469b8c-6ce2-41dd-b005-71360286a5ee None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:00.273315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42469b8c-6ce2-41dd-b005-71360286a5ee None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:00.276872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55439526-e400-4c8b-8f32-39f62d6ca760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:00.277499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55439526-e400-4c8b-8f32-39f62d6ca760 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:00.277804 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55439526-e400-4c8b-8f32-39f62d6ca760 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:00.278073 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55439526-e400-4c8b-8f32-39f62d6ca760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:00.287145 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55439526-e400-4c8b-8f32-39f62d6ca760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:00.288584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55439526-e400-4c8b-8f32-39f62d6ca760 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:00.298062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 725/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:11:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:02.883920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e84d5-ffef-43c8-bc93-64022f5d2fd0 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:02.884584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e84d5-ffef-43c8-bc93-64022f5d2fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:02.884584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e84d5-ffef-43c8-bc93-64022f5d2fd0 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:02.885913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:02.891343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd508074-8704-401a-8f9f-593bbeaa168d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:02.891821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd508074-8704-401a-8f9f-593bbeaa168d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:02.891965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd508074-8704-401a-8f9f-593bbeaa168d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:02.910289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-cd508074-8704-401a-8f9f-593bbeaa168d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:02.911482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cd508074-8704-401a-8f9f-593bbeaa168d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:02.919622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:02.957748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e84d5-ffef-43c8-bc93-64022f5d2fd0 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:02.968749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e84d5-ffef-43c8-bc93-64022f5d2fd0 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:02.972832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd2bd4-2b1a-497b-bc3f-0d6df948c3a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:02.973473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd2bd4-2b1a-497b-bc3f-0d6df948c3a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:02.973782 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd2bd4-2b1a-497b-bc3f-0d6df948c3a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:02.974062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd2bd4-2b1a-497b-bc3f-0d6df948c3a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:02.982929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd2bd4-2b1a-497b-bc3f-0d6df948c3a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:02.984361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd2bd4-2b1a-497b-bc3f-0d6df948c3a8 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:02.994989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 723/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:11:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:11:05.355448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411dffd-367f-4e9b-b0ee-6e0efb89fcfa None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:05.355973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411dffd-367f-4e9b-b0ee-6e0efb89fcfa None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:05.356082 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411dffd-367f-4e9b-b0ee-6e0efb89fcfa None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:05.357396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:05.363400 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d98e7-e8fe-4219-917b-acf5fe67be25 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:05.363829 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d98e7-e8fe-4219-917b-acf5fe67be25 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:05.363940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d98e7-e8fe-4219-917b-acf5fe67be25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:05.381353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-599d98e7-e8fe-4219-917b-acf5fe67be25 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:05.382649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-599d98e7-e8fe-4219-917b-acf5fe67be25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:05.391060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:05.430930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411dffd-367f-4e9b-b0ee-6e0efb89fcfa None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:05.442091 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411dffd-367f-4e9b-b0ee-6e0efb89fcfa None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:05.445531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616e53c-745b-4764-9314-d733ef8a1378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:05.446193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616e53c-745b-4764-9314-d733ef8a1378 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:05.446464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616e53c-745b-4764-9314-d733ef8a1378 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:05.446730 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616e53c-745b-4764-9314-d733ef8a1378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:05.455884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616e53c-745b-4764-9314-d733ef8a1378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:05.458228 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616e53c-745b-4764-9314-d733ef8a1378 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:05.467507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 728/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:11:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:07.802337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313a3bb-89d2-493a-8797-89d27baf9d89 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:07.802885 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313a3bb-89d2-493a-8797-89d27baf9d89 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:07.802885 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313a3bb-89d2-493a-8797-89d27baf9d89 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:07.804595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:07.810651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8da372-6dbf-4821-be70-30c511f6e4dd None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:07.811389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8da372-6dbf-4821-be70-30c511f6e4dd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:07.811550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8da372-6dbf-4821-be70-30c511f6e4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:07.830010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-bc8da372-6dbf-4821-be70-30c511f6e4dd None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:07.831275 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bc8da372-6dbf-4821-be70-30c511f6e4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:07.839015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:07.876412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313a3bb-89d2-493a-8797-89d27baf9d89 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:07.888962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313a3bb-89d2-493a-8797-89d27baf9d89 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:07.892422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595db682-3f71-492e-80b2-9d4eae45952d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:07.893055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595db682-3f71-492e-80b2-9d4eae45952d service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:07.893336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595db682-3f71-492e-80b2-9d4eae45952d service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:07.893610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595db682-3f71-492e-80b2-9d4eae45952d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:07.902394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595db682-3f71-492e-80b2-9d4eae45952d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:07.903812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595db682-3f71-492e-80b2-9d4eae45952d service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:07.913880 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 726/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:11:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:10.362143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3297a5f-091b-46ab-8c69-2ce1c6c55a26 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:10.362703 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3297a5f-091b-46ab-8c69-2ce1c6c55a26 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:10.362703 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3297a5f-091b-46ab-8c69-2ce1c6c55a26 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:10.364164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:10.369690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a7911-d74f-4e21-bb30-d67aeb8e3490 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:10.370094 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a7911-d74f-4e21-bb30-d67aeb8e3490 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:10.370344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a7911-d74f-4e21-bb30-d67aeb8e3490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:10.388246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-315a7911-d74f-4e21-bb30-d67aeb8e3490 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:10.389442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-315a7911-d74f-4e21-bb30-d67aeb8e3490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:10.398053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:10.435680 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3297a5f-091b-46ab-8c69-2ce1c6c55a26 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:10.447494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3297a5f-091b-46ab-8c69-2ce1c6c55a26 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:10.451245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a125fc-3f46-405b-8d4e-91061cc21be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:10.451883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a125fc-3f46-405b-8d4e-91061cc21be1 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:10.452163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a125fc-3f46-405b-8d4e-91061cc21be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:10.452438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a125fc-3f46-405b-8d4e-91061cc21be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:10.463045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a125fc-3f46-405b-8d4e-91061cc21be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:10.464496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a125fc-3f46-405b-8d4e-91061cc21be1 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:10.474486 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 731/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:12.780188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae77074-fe4e-4187-b85f-72a124be6bcf None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:12.780757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae77074-fe4e-4187-b85f-72a124be6bcf None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:12.780757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae77074-fe4e-4187-b85f-72a124be6bcf None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:12.782161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:12.787671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3929c5e-458b-4f3e-945c-d4cf12c09dd7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:12.788121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3929c5e-458b-4f3e-945c-d4cf12c09dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:12.788226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3929c5e-458b-4f3e-945c-d4cf12c09dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:12.794451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=70 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:12.807151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e3929c5e-458b-4f3e-945c-d4cf12c09dd7 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:12.808337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e3929c5e-458b-4f3e-945c-d4cf12c09dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:12.816723 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:12.855608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae77074-fe4e-4187-b85f-72a124be6bcf None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:12.861100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=73 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:12.868945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae77074-fe4e-4187-b85f-72a124be6bcf None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:12.873187 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d722a30-8be8-4d43-93b8-5f6b5dec7a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:12.873892 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d722a30-8be8-4d43-93b8-5f6b5dec7a57 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:12.874169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d722a30-8be8-4d43-93b8-5f6b5dec7a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:12.874438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d722a30-8be8-4d43-93b8-5f6b5dec7a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:12.885659 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d722a30-8be8-4d43-93b8-5f6b5dec7a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:12.887293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d722a30-8be8-4d43-93b8-5f6b5dec7a57 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:12.901799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 729/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:15.207263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86082196-33a8-45da-8c3c-6f0df3e5f85e None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:15.207834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86082196-33a8-45da-8c3c-6f0df3e5f85e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:15.207834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86082196-33a8-45da-8c3c-6f0df3e5f85e None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:15.209176 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:11:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:11:15.214738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb7b5c-cf73-4442-9f7f-a8832c79bbff None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:15.215187 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb7b5c-cf73-4442-9f7f-a8832c79bbff None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:15.215318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb7b5c-cf73-4442-9f7f-a8832c79bbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:15.235082 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e4eb7b5c-cf73-4442-9f7f-a8832c79bbff None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:15.236296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e4eb7b5c-cf73-4442-9f7f-a8832c79bbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:15.244908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:15.283154 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86082196-33a8-45da-8c3c-6f0df3e5f85e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:15.295539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86082196-33a8-45da-8c3c-6f0df3e5f85e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:15.299370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f34649-5d4d-4adf-b96b-617033440d27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:15.300596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f34649-5d4d-4adf-b96b-617033440d27 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:15.300885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f34649-5d4d-4adf-b96b-617033440d27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:15.301156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f34649-5d4d-4adf-b96b-617033440d27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:15.310784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f34649-5d4d-4adf-b96b-617033440d27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:15.312377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f34649-5d4d-4adf-b96b-617033440d27 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:15.323200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 734/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:11:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:17.607742 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f08d4-25ad-4255-a791-154f9480d63d None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:17.608289 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f08d4-25ad-4255-a791-154f9480d63d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:17.608289 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f08d4-25ad-4255-a791-154f9480d63d None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:17.609564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:17.615819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60ba12-f7ee-4280-b0b3-817a12eee193 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:17.616262 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60ba12-f7ee-4280-b0b3-817a12eee193 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:17.616360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60ba12-f7ee-4280-b0b3-817a12eee193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:17.635074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1b60ba12-f7ee-4280-b0b3-817a12eee193 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:17.636264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1b60ba12-f7ee-4280-b0b3-817a12eee193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:17.644325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:17.681074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f08d4-25ad-4255-a791-154f9480d63d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:17.693036 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f08d4-25ad-4255-a791-154f9480d63d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:17.696851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed1976-b92b-4da0-b563-13486c7031e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:17.697483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed1976-b92b-4da0-b563-13486c7031e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:17.697782 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed1976-b92b-4da0-b563-13486c7031e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:17.698049 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed1976-b92b-4da0-b563-13486c7031e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:17.708040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed1976-b92b-4da0-b563-13486c7031e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:17.709463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed1976-b92b-4da0-b563-13486c7031e8 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:17.718700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 732/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:11:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:19.958540 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1463e-e354-4f67-baa1-4983da4ec828 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:19.959105 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1463e-e354-4f67-baa1-4983da4ec828 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:19.959105 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1463e-e354-4f67-baa1-4983da4ec828 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:19.960492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:19.965846 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca776f5a-6ded-408e-bd9f-92bd6b88b2ab None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:19.966337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca776f5a-6ded-408e-bd9f-92bd6b88b2ab None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:19.966443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca776f5a-6ded-408e-bd9f-92bd6b88b2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:19.983793 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ca776f5a-6ded-408e-bd9f-92bd6b88b2ab None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:19.984998 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ca776f5a-6ded-408e-bd9f-92bd6b88b2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:19.993559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:20.029741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1463e-e354-4f67-baa1-4983da4ec828 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:20.041067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1463e-e354-4f67-baa1-4983da4ec828 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:20.044827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284aa53a-d538-40ae-8a4b-84a899245d6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:20.045450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284aa53a-d538-40ae-8a4b-84a899245d6c service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:20.045746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284aa53a-d538-40ae-8a4b-84a899245d6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:20.046021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284aa53a-d538-40ae-8a4b-84a899245d6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:20.055884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284aa53a-d538-40ae-8a4b-84a899245d6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:20.057361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284aa53a-d538-40ae-8a4b-84a899245d6c service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:20.072724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 737/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:11:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:29.978592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=17 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:30.063102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=17 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:53.746176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f83e21-7ad1-41c7-8471-dd3e05946638 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:53.746812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f83e21-7ad1-41c7-8471-dd3e05946638 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:53.746812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f83e21-7ad1-41c7-8471-dd3e05946638 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:53.748203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 734/1471] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:53.754430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac11ba-b3ba-4459-9b28-f74291b90c61 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:53.755623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac11ba-b3ba-4459-9b28-f74291b90c61 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:53.755623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac11ba-b3ba-4459-9b28-f74291b90c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:53.774509 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1eac11ba-b3ba-4459-9b28-f74291b90c61 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:53.776428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1eac11ba-b3ba-4459-9b28-f74291b90c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:53.784924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 738/1472] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:11:53.832447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f959b9e-ec71-4498-b7a4-f2d55308d67f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:53.832798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f959b9e-ec71-4498-b7a4-f2d55308d67f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:53.833119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f959b9e-ec71-4498-b7a4-f2d55308d67f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:53.834657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 735/1473] 10.222.0.46 () {54 vars in 962 bytes} [Thu May 26 18:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:53.839262 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ada60e-f472-4be7-80f5-3bae26e82989 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:53.839548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ada60e-f472-4be7-80f5-3bae26e82989 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:53.839829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ada60e-f472-4be7-80f5-3bae26e82989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:53.857981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-56ada60e-f472-4be7-80f5-3bae26e82989 None None] MFA Rules not processed for user `4a67120485f74f8bb0ff188cd3abd21c`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:53.859172 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-56ada60e-f472-4be7-80f5-3bae26e82989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:53.866309 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 739/1474] 10.222.0.46 () {58 vars in 1059 bytes} [Thu May 26 18:11:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:53.873273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f959b9e-ec71-4498-b7a4-f2d55308d67f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:53.883783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f959b9e-ec71-4498-b7a4-f2d55308d67f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:53.887290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bd03f-d4e5-4bf5-84c8-0f10abd18e16 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a67120485f74f8bb0ff188cd3abd21c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:53.887911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bd03f-d4e5-4bf5-84c8-0f10abd18e16 service glance] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:53.888192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bd03f-d4e5-4bf5-84c8-0f10abd18e16 service glance] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:53.888464 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bd03f-d4e5-4bf5-84c8-0f10abd18e16 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:53.897493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bd03f-d4e5-4bf5-84c8-0f10abd18e16 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:53.898958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bd03f-d4e5-4bf5-84c8-0f10abd18e16 service glance] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:53.908706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 736/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:11:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:11:56.262510 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240235b4-fc4b-435d-ba5c-b1f832a7dee9 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:56.263143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240235b4-fc4b-435d-ba5c-b1f832a7dee9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:56.263143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240235b4-fc4b-435d-ba5c-b1f832a7dee9 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:56.264428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 740/1476] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:56.270101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cd07e-1ae3-44ef-8256-bf044f069332 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:56.270585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cd07e-1ae3-44ef-8256-bf044f069332 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:56.270679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cd07e-1ae3-44ef-8256-bf044f069332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:56.290653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-397cd07e-1ae3-44ef-8256-bf044f069332 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:56.292135 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-397cd07e-1ae3-44ef-8256-bf044f069332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:56.300653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 737/1477] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:56.323914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240235b4-fc4b-435d-ba5c-b1f832a7dee9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:56.336620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240235b4-fc4b-435d-ba5c-b1f832a7dee9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:56.340978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f1977-4f09-44c6-8bda-b839d3d9dfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:56.341625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f1977-4f09-44c6-8bda-b839d3d9dfb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:56.341936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f1977-4f09-44c6-8bda-b839d3d9dfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:56.342210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f1977-4f09-44c6-8bda-b839d3d9dfb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:56.342941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f1977-4f09-44c6-8bda-b839d3d9dfb4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:56.344747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-728f1977-4f09-44c6-8bda-b839d3d9dfb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:11:56.344747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: warnings.warn(msg) May 26 18:11:56.344747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: May 26 18:11:56.346489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f1977-4f09-44c6-8bda-b839d3d9dfb4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:56.366838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 741/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 18:11:56 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:58.654192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd99f22-37f4-45f2-a344-f18a24dee40a None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:58.654718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd99f22-37f4-45f2-a344-f18a24dee40a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:58.654835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd99f22-37f4-45f2-a344-f18a24dee40a None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:58.656131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 738/1479] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:58.662286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e41ea-9f84-407d-aa46-799d752fafd6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:58.662698 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e41ea-9f84-407d-aa46-799d752fafd6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:58.662894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e41ea-9f84-407d-aa46-799d752fafd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:58.681797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-be2e41ea-9f84-407d-aa46-799d752fafd6 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:58.683016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-be2e41ea-9f84-407d-aa46-799d752fafd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:58.690960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 742/1480] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:11:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:58.743738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd99f22-37f4-45f2-a344-f18a24dee40a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:58.758262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd99f22-37f4-45f2-a344-f18a24dee40a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:58.763860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16322603-1106-4647-a497-aef8c76a5c83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:58.764495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16322603-1106-4647-a497-aef8c76a5c83 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:58.764774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16322603-1106-4647-a497-aef8c76a5c83 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:58.765046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16322603-1106-4647-a497-aef8c76a5c83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:58.773975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16322603-1106-4647-a497-aef8c76a5c83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:58.775555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16322603-1106-4647-a497-aef8c76a5c83 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:58.785567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 739/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:11:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:01.234470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952cdff-14b1-4fae-a2e5-ff6effb49a13 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.235063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952cdff-14b1-4fae-a2e5-ff6effb49a13 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.235063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952cdff-14b1-4fae-a2e5-ff6effb49a13 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.236386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 743/1482] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:12:01.241771 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cb146-6e26-47f4-81cd-14f020c7ada6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.242229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cb146-6e26-47f4-81cd-14f020c7ada6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.242328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cb146-6e26-47f4-81cd-14f020c7ada6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.260543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e54cb146-6e26-47f4-81cd-14f020c7ada6 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:01.261758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e54cb146-6e26-47f4-81cd-14f020c7ada6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:01.270289 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 740/1483] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:01.307737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952cdff-14b1-4fae-a2e5-ff6effb49a13 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:01.319677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952cdff-14b1-4fae-a2e5-ff6effb49a13 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:01.323293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da7b28-f91c-451a-9fa9-ff91b4652d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:01.323939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da7b28-f91c-451a-9fa9-ff91b4652d96 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.324214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da7b28-f91c-451a-9fa9-ff91b4652d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.324484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da7b28-f91c-451a-9fa9-ff91b4652d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.335451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da7b28-f91c-451a-9fa9-ff91b4652d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:01.336981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da7b28-f91c-451a-9fa9-ff91b4652d96 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:01.346808 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 744/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:12:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:03.657780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab16e7-ec3a-4d43-bd19-4bddfa85d5a4 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:03.658349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab16e7-ec3a-4d43-bd19-4bddfa85d5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:03.658349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab16e7-ec3a-4d43-bd19-4bddfa85d5a4 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:03.659749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 741/1485] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:12:03.665078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06681c8-ca77-4430-a7fe-18f159353ced None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:03.665470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06681c8-ca77-4430-a7fe-18f159353ced None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:03.665634 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06681c8-ca77-4430-a7fe-18f159353ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:03.683603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a06681c8-ca77-4430-a7fe-18f159353ced None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:03.684816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a06681c8-ca77-4430-a7fe-18f159353ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:03.693231 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 745/1486] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:03.731203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab16e7-ec3a-4d43-bd19-4bddfa85d5a4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:03.742353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab16e7-ec3a-4d43-bd19-4bddfa85d5a4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:03.746576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302fb5c-9527-4a51-9b28-2acb5a9ed0fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:03.747227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302fb5c-9527-4a51-9b28-2acb5a9ed0fd service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:03.747511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302fb5c-9527-4a51-9b28-2acb5a9ed0fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:03.747786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302fb5c-9527-4a51-9b28-2acb5a9ed0fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:03.758985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302fb5c-9527-4a51-9b28-2acb5a9ed0fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:03.760439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302fb5c-9527-4a51-9b28-2acb5a9ed0fd service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:03.777225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 742/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:12:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:06.212066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e675a-8a25-40d2-951f-b7f2246a5c02 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:06.212660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e675a-8a25-40d2-951f-b7f2246a5c02 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:06.212660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e675a-8a25-40d2-951f-b7f2246a5c02 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:06.214032 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 746/1488] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:12:06.219154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdbc9c-8aec-4488-b8b3-9e6873219aa9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:06.219596 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdbc9c-8aec-4488-b8b3-9e6873219aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:06.219704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdbc9c-8aec-4488-b8b3-9e6873219aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:06.246736 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2dbdbc9c-8aec-4488-b8b3-9e6873219aa9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:06.247954 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2dbdbc9c-8aec-4488-b8b3-9e6873219aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:06.256418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 743/1489] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:06.297614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e675a-8a25-40d2-951f-b7f2246a5c02 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:06.308548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e675a-8a25-40d2-951f-b7f2246a5c02 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:06.312097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee058de-e1c3-4e47-83b7-f65d421e2e78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:06.312731 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee058de-e1c3-4e47-83b7-f65d421e2e78 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:06.313042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee058de-e1c3-4e47-83b7-f65d421e2e78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:06.313313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee058de-e1c3-4e47-83b7-f65d421e2e78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:06.323098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee058de-e1c3-4e47-83b7-f65d421e2e78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:06.324555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee058de-e1c3-4e47-83b7-f65d421e2e78 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:06.334768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 747/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:12:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:08.840792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08274658-f3f1-41f5-b3b9-f706a1a44ebe None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:08.841357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08274658-f3f1-41f5-b3b9-f706a1a44ebe None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:08.841357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08274658-f3f1-41f5-b3b9-f706a1a44ebe None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:08.842763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 744/1491] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 18:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:12:08.848654 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c60b8a-2733-4ef7-890d-d2dc45bc9f22 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:08.849085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c60b8a-2733-4ef7-890d-d2dc45bc9f22 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:08.849187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c60b8a-2733-4ef7-890d-d2dc45bc9f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:08.867430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-35c60b8a-2733-4ef7-890d-d2dc45bc9f22 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:08.868622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-35c60b8a-2733-4ef7-890d-d2dc45bc9f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:08.876279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 748/1492] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 18:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:08.924939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08274658-f3f1-41f5-b3b9-f706a1a44ebe None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:08.935977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08274658-f3f1-41f5-b3b9-f706a1a44ebe None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:08.939499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc79208-2884-4c62-a047-86f59ace07f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:08.940184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc79208-2884-4c62-a047-86f59ace07f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:08.940460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc79208-2884-4c62-a047-86f59ace07f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:08.940848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc79208-2884-4c62-a047-86f59ace07f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:08.953964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc79208-2884-4c62-a047-86f59ace07f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:08.953964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc79208-2884-4c62-a047-86f59ace07f8 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:08.961881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 745/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:12:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:18.861555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=33 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:18.957339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=33 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:50.446309 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332d3cc-3e32-49fb-b771-991470a9d0eb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:50.446904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332d3cc-3e32-49fb-b771-991470a9d0eb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:50.446904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332d3cc-3e32-49fb-b771-991470a9d0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:50.465491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3332d3cc-3e32-49fb-b771-991470a9d0eb None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:50.466764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3332d3cc-3e32-49fb-b771-991470a9d0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:50.474726 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 749/1494] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:13:50.483866 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96c126-0a88-40e2-9227-9454ba59f211 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:50.484354 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96c126-0a88-40e2-9227-9454ba59f211 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:50.484449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96c126-0a88-40e2-9227-9454ba59f211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:50.502932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bd96c126-0a88-40e2-9227-9454ba59f211 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:50.504123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bd96c126-0a88-40e2-9227-9454ba59f211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:50.512232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 746/1495] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:13:50.517850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332d3cc-3e32-49fb-b771-991470a9d0eb None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:50.527444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332d3cc-3e32-49fb-b771-991470a9d0eb None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:50.531866 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b418281-8bcb-42ae-9766-91adc598fa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:50.532511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b418281-8bcb-42ae-9766-91adc598fa74 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:50.532791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b418281-8bcb-42ae-9766-91adc598fa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:50.533061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b418281-8bcb-42ae-9766-91adc598fa74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:50.533818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b418281-8bcb-42ae-9766-91adc598fa74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:13:50.534125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b418281-8bcb-42ae-9766-91adc598fa74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:50.535158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-3b418281-8bcb-42ae-9766-91adc598fa74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 18:13:50.535158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: warnings.warn(msg) May 26 18:13:50.535158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: May 26 18:13:50.536032 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b418281-8bcb-42ae-9766-91adc598fa74 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:50.548384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 750/1496] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:13:50.556483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96c126-0a88-40e2-9227-9454ba59f211 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:50.566509 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96c126-0a88-40e2-9227-9454ba59f211 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:50.571124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75135b-3594-43d1-9af8-4ee6a98f5bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:50.571774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75135b-3594-43d1-9af8-4ee6a98f5bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:50.572058 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75135b-3594-43d1-9af8-4ee6a98f5bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:50.572387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75135b-3594-43d1-9af8-4ee6a98f5bb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:50.573084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75135b-3594-43d1-9af8-4ee6a98f5bb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:50.574889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-8a75135b-3594-43d1-9af8-4ee6a98f5bb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:13:50.574889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: warnings.warn(msg) May 26 18:13:50.574889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: May 26 18:13:50.576079 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75135b-3594-43d1-9af8-4ee6a98f5bb2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:50.612554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 747/1497] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:13:50 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:13:50.616735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b418281-8bcb-42ae-9766-91adc598fa74 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:50.627257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b418281-8bcb-42ae-9766-91adc598fa74 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:50.682586 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446b31f-4352-466a-8260-9a568b55972f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:50.683281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446b31f-4352-466a-8260-9a568b55972f admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:50.683563 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446b31f-4352-466a-8260-9a568b55972f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:50.683839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446b31f-4352-466a-8260-9a568b55972f admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:50.684524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446b31f-4352-466a-8260-9a568b55972f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:50.685553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-e446b31f-4352-466a-8260-9a568b55972f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:13:50.685553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: warnings.warn(msg) May 26 18:13:50.685553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: May 26 18:13:50.686791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446b31f-4352-466a-8260-9a568b55972f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:50.707019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 751/1498] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:13:50 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:13:50.711175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75135b-3594-43d1-9af8-4ee6a98f5bb2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:50.721895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75135b-3594-43d1-9af8-4ee6a98f5bb2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:50.726372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd319ab0-a403-4790-94b3-64c651fdfeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:50.727024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd319ab0-a403-4790-94b3-64c651fdfeb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:50.727307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd319ab0-a403-4790-94b3-64c651fdfeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:50.727581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd319ab0-a403-4790-94b3-64c651fdfeb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:50.728294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd319ab0-a403-4790-94b3-64c651fdfeb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:50.729302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-dd319ab0-a403-4790-94b3-64c651fdfeb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 18:13:50.729302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: warnings.warn(msg) May 26 18:13:50.729302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: May 26 18:13:50.730212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd319ab0-a403-4790-94b3-64c651fdfeb5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:50.743064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 748/1499] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:13:50.747160 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446b31f-4352-466a-8260-9a568b55972f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:50.757405 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446b31f-4352-466a-8260-9a568b55972f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:50.761991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc886f7-7876-4f63-8c80-d7028c8ccca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:50.762703 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc886f7-7876-4f63-8c80-d7028c8ccca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:50.762985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc886f7-7876-4f63-8c80-d7028c8ccca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:50.763289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc886f7-7876-4f63-8c80-d7028c8ccca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:50.764007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc886f7-7876-4f63-8c80-d7028c8ccca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:50.765039 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-bdc886f7-7876-4f63-8c80-d7028c8ccca6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 18:13:50.765039 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: warnings.warn(msg) May 26 18:13:50.765039 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: May 26 18:13:50.765944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc886f7-7876-4f63-8c80-d7028c8ccca6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:50.779799 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 752/1500] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:13:50.784438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd319ab0-a403-4790-94b3-64c651fdfeb5 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:50.794472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd319ab0-a403-4790-94b3-64c651fdfeb5 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:50.799551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bd2b8-42fd-492d-b81a-01cbe26178fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:50.800416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bd2b8-42fd-492d-b81a-01cbe26178fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:50.800696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bd2b8-42fd-492d-b81a-01cbe26178fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:50.800968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bd2b8-42fd-492d-b81a-01cbe26178fe admin admin] PATH_INFO: `/v3/projects/680f540b9578404b901a7f9255255e85/users/fb0b9a0a79e8451796046b314e758d11/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:50.816241 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bd2b8-42fd-492d-b81a-01cbe26178fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=680f540b9578404b901a7f9255255e85, user_id=fb0b9a0a79e8451796046b314e758d11, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:50.817269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-d49bd2b8-42fd-492d-b81a-01cbe26178fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:13:50.817269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: warnings.warn(msg) May 26 18:13:50.817269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: May 26 18:13:50.819282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bd2b8-42fd-492d-b81a-01cbe26178fe admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:50.832095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 749/1501] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:13:50 2022] PUT /identity/v3/projects/680f540b9578404b901a7f9255255e85/users/fb0b9a0a79e8451796046b314e758d11/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:13:50.843183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056974c-885a-44b9-b99b-9482e23fd64e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:50.843469 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056974c-885a-44b9-b99b-9482e23fd64e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:50.843744 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056974c-885a-44b9-b99b-9482e23fd64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:50.870580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9056974c-885a-44b9-b99b-9482e23fd64e None None] MFA Rules not processed for user `fb0b9a0a79e8451796046b314e758d11`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:50.871792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9056974c-885a-44b9-b99b-9482e23fd64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:50.935168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 753/1502] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:13:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:13:50.948318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9780e5-0cc9-43aa-bf09-f4c9c294d425 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:50.948610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9780e5-0cc9-43aa-bf09-f4c9c294d425 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:50.948893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9780e5-0cc9-43aa-bf09-f4c9c294d425 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:50.950291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 750/1503] 10.222.0.46 () {54 vars in 963 bytes} [Thu May 26 18:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:13:50.955080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce9d81-bcce-4fd1-8bea-0f2cf2565ced None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:50.955368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce9d81-bcce-4fd1-8bea-0f2cf2565ced None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:50.955640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce9d81-bcce-4fd1-8bea-0f2cf2565ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:50.973700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-39ce9d81-bcce-4fd1-8bea-0f2cf2565ced None None] MFA Rules not processed for user `8c79367697f7451fb804b8641444f6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:50.974967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-39ce9d81-bcce-4fd1-8bea-0f2cf2565ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:51.017669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:51.025771 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 754/1504] 10.222.0.46 () {58 vars in 1060 bytes} [Thu May 26 18:13:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:13:51.136120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9780e5-0cc9-43aa-bf09-f4c9c294d425 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:51.147135 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9780e5-0cc9-43aa-bf09-f4c9c294d425 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:51.150644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b999a6-b393-4ca2-a0d0-b3419926824e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:51.151268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b999a6-b393-4ca2-a0d0-b3419926824e service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:51.151547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b999a6-b393-4ca2-a0d0-b3419926824e service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:51.151818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b999a6-b393-4ca2-a0d0-b3419926824e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:51.160474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b999a6-b393-4ca2-a0d0-b3419926824e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:51.161926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b999a6-b393-4ca2-a0d0-b3419926824e service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:51.170739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 751/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:13:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:13:51.213532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3fda-1081-4b4a-bfab-530d854305db None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:51.213852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3fda-1081-4b4a-bfab-530d854305db None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:51.214134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3fda-1081-4b4a-bfab-530d854305db None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:51.215420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 755/1506] 10.222.0.46 () {52 vars in 829 bytes} [Thu May 26 18:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:13:51.218990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b999a6-b393-4ca2-a0d0-b3419926824e service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:51.281912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b999a6-b393-4ca2-a0d0-b3419926824e service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:51.287288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:51.288159 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:51.288441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:51.288716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] PATH_INFO: `/v3/users/fb0b9a0a79e8451796046b314e758d11` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:51.289924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb0b9a0a79e8451796046b314e758d11)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:51.290943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:13:51.290943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: warnings.warn(msg) May 26 18:13:51.290943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: May 26 18:13:51.292458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:51.358030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb0b9a0a79e8451796046b314e758d11'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:51.367965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb0b9a0a79e8451796046b314e758d11'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:51.387367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb0b9a0a79e8451796046b314e758d11'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:51.388511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb0b9a0a79e8451796046b314e758d11'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:51.400623 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:51.404967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb0b9a0a79e8451796046b314e758d11'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:51.413173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 752/1507] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:13:51 2022] DELETE /identity/v3/users/fb0b9a0a79e8451796046b314e758d11 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:13:51.417846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff473a-21f5-4994-ab40-9bbe7f4fc6f7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:51.418139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff473a-21f5-4994-ab40-9bbe7f4fc6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:51.418417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff473a-21f5-4994-ab40-9bbe7f4fc6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:51.437004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c1ff473a-21f5-4994-ab40-9bbe7f4fc6f7 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:51.438827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c1ff473a-21f5-4994-ab40-9bbe7f4fc6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:51.498365 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 756/1508] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:13:51.508896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:51.510392 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:51.567488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48260236-65b9-48e4-b1bc-4f3f67686dec admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:51.571137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753c90d-2724-4ae8-b514-ffbd69db6b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:51.571804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753c90d-2724-4ae8-b514-ffbd69db6b38 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:51.572090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753c90d-2724-4ae8-b514-ffbd69db6b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:51.572364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753c90d-2724-4ae8-b514-ffbd69db6b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:51.581886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753c90d-2724-4ae8-b514-ffbd69db6b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:51.583322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753c90d-2724-4ae8-b514-ffbd69db6b38 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:51.592942 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 753/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:13:51.916562 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff473a-21f5-4994-ab40-9bbe7f4fc6f7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:51.918372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c1ff473a-21f5-4994-ab40-9bbe7f4fc6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:51.930574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff473a-21f5-4994-ab40-9bbe7f4fc6f7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:51.935383 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:51.936222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:51.936499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:51.936765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] PATH_INFO: `/v3/projects/680f540b9578404b901a7f9255255e85` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:51.938599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] RBAC: Authorizing `identity:delete_project(project_id=680f540b9578404b901a7f9255255e85)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:51.939736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:13:51.939736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: warnings.warn(msg) May 26 18:13:51.939736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: May 26 18:13:51.940836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:52.001155 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '680f540b9578404b901a7f9255255e85', 'request_id': 'req-1e500392-90f7-483b-bf29-c5438c56d91b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:52.014343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '680f540b9578404b901a7f9255255e85', 'request_id': 'req-1e500392-90f7-483b-bf29-c5438c56d91b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:52.021190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '680f540b9578404b901a7f9255255e85', 'request_id': 'req-1e500392-90f7-483b-bf29-c5438c56d91b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:52.040498 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] The token cache is being invalidate because project 680f540b9578404b901a7f9255255e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:13:52.040785 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:52.041421 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1e500392-90f7-483b-bf29-c5438c56d91b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:52.043470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 757/1510] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:13:51 2022] DELETE /identity/v3/projects/680f540b9578404b901a7f9255255e85 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:14:01.588800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=29 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:14:02.036538 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:37.342690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecf960-cc1e-45a5-b791-d959dbb1221a None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:37.343199 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecf960-cc1e-45a5-b791-d959dbb1221a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:37.343309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecf960-cc1e-45a5-b791-d959dbb1221a None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:37.344667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:15:37.349893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96827b-7b1a-4330-ae80-82a747ea49c3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:37.350365 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96827b-7b1a-4330-ae80-82a747ea49c3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:37.350472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96827b-7b1a-4330-ae80-82a747ea49c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:37.368501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ed96827b-7b1a-4330-ae80-82a747ea49c3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:37.369715 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ed96827b-7b1a-4330-ae80-82a747ea49c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:37.428295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:37.599024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecf960-cc1e-45a5-b791-d959dbb1221a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:37.600760 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-96ecf960-cc1e-45a5-b791-d959dbb1221a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:37.675745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecf960-cc1e-45a5-b791-d959dbb1221a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:37.681097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf806a0-64af-49ab-9e4d-4d36dee9d5eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:37.681761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf806a0-64af-49ab-9e4d-4d36dee9d5eb service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:37.682046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf806a0-64af-49ab-9e4d-4d36dee9d5eb service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:37.682323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf806a0-64af-49ab-9e4d-4d36dee9d5eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:37.690414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf806a0-64af-49ab-9e4d-4d36dee9d5eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:37.691997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf806a0-64af-49ab-9e4d-4d36dee9d5eb service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:37.701273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 755/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:38.587811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e11094-254f-4104-b819-75214e84dc18 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:38.588336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e11094-254f-4104-b819-75214e84dc18 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:38.588336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e11094-254f-4104-b819-75214e84dc18 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:38.589652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:15:38.594470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f08b7-cd07-4e95-9d01-f4a022d530d4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:38.594861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f08b7-cd07-4e95-9d01-f4a022d530d4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:38.595032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f08b7-cd07-4e95-9d01-f4a022d530d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:38.613448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-740f08b7-cd07-4e95-9d01-f4a022d530d4 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:38.614671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-740f08b7-cd07-4e95-9d01-f4a022d530d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:38.622390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:38.775112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e11094-254f-4104-b819-75214e84dc18 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:38.790013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e11094-254f-4104-b819-75214e84dc18 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:38.795693 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52ba28-8927-435f-9c42-6245de7323d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:38.796327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52ba28-8927-435f-9c42-6245de7323d3 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:38.796610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52ba28-8927-435f-9c42-6245de7323d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:38.796885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52ba28-8927-435f-9c42-6245de7323d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:38.805718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52ba28-8927-435f-9c42-6245de7323d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:38.807362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52ba28-8927-435f-9c42-6245de7323d3 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:38.817074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 760/1516] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:39.204010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6f929-d0de-45f8-9acb-67ff32eea56a None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:39.204322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6f929-d0de-45f8-9acb-67ff32eea56a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:39.204600 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6f929-d0de-45f8-9acb-67ff32eea56a None None] PATH_INFO: `/v3` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:39.206130 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 757/1517] 10.222.0.42 () {54 vars in 925 bytes} [Thu May 26 18:15:39 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:15:39.212739 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4b961-7474-4d5f-be11-6f19e6c03c1c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:39.213032 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4b961-7474-4d5f-be11-6f19e6c03c1c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:39.213314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4b961-7474-4d5f-be11-6f19e6c03c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:39.231273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-54a4b961-7474-4d5f-be11-6f19e6c03c1c None None] MFA Rules not processed for user `7ee40bcb7905490c9ad3e39e7a93b122`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:39.232478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-54a4b961-7474-4d5f-be11-6f19e6c03c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:39.294678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 761/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Thu May 26 18:15:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:43.246178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98806751-ca9a-422f-ac27-c6f6f1a933e8 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:43.246738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98806751-ca9a-422f-ac27-c6f6f1a933e8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:43.246738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98806751-ca9a-422f-ac27-c6f6f1a933e8 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:43.248075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 758/1519] 10.222.0.46 () {54 vars in 960 bytes} [Thu May 26 18:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:15:43.253667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ea9b5-5f02-4d79-b1ad-a787698f2802 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:43.254095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ea9b5-5f02-4d79-b1ad-a787698f2802 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:43.254294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ea9b5-5f02-4d79-b1ad-a787698f2802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:43.272386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-252ea9b5-5f02-4d79-b1ad-a787698f2802 None None] MFA Rules not processed for user `7ee40bcb7905490c9ad3e39e7a93b122`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:43.273615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-252ea9b5-5f02-4d79-b1ad-a787698f2802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:43.282175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 762/1520] 10.222.0.46 () {58 vars in 1057 bytes} [Thu May 26 18:15:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:43.290402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98806751-ca9a-422f-ac27-c6f6f1a933e8 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:43.304505 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98806751-ca9a-422f-ac27-c6f6f1a933e8 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:43.310087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6de1fe-a70b-42d1-8595-2a4034120ae0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:43.310747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6de1fe-a70b-42d1-8595-2a4034120ae0 service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:43.311034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6de1fe-a70b-42d1-8595-2a4034120ae0 service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:43.311305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6de1fe-a70b-42d1-8595-2a4034120ae0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:43.321118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6de1fe-a70b-42d1-8595-2a4034120ae0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:43.323867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6de1fe-a70b-42d1-8595-2a4034120ae0 service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:43.335155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 759/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:15:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:50.114546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfbbb4-cf65-4b6a-a545-a7e967f1e68a None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:50.115205 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfbbb4-cf65-4b6a-a545-a7e967f1e68a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:50.115205 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfbbb4-cf65-4b6a-a545-a7e967f1e68a None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:50.116463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:15:50.121715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378b5a3-51e1-45d7-b3a4-bcd207eaf854 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:50.122193 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378b5a3-51e1-45d7-b3a4-bcd207eaf854 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:50.122292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378b5a3-51e1-45d7-b3a4-bcd207eaf854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:50.140874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4378b5a3-51e1-45d7-b3a4-bcd207eaf854 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:50.142108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4378b5a3-51e1-45d7-b3a4-bcd207eaf854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:50.150516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:50.306314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfbbb4-cf65-4b6a-a545-a7e967f1e68a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:50.322177 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfbbb4-cf65-4b6a-a545-a7e967f1e68a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:50.327895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f202a5-ea07-434e-8139-1f0c39295d44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:50.328540 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f202a5-ea07-434e-8139-1f0c39295d44 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:50.328820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f202a5-ea07-434e-8139-1f0c39295d44 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:50.329092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f202a5-ea07-434e-8139-1f0c39295d44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:50.337743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f202a5-ea07-434e-8139-1f0c39295d44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:50.339377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f202a5-ea07-434e-8139-1f0c39295d44 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:50.349258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 764/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:51.222296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f26c14-1b3a-44e9-8190-d80369b87c8b None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:51.222861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f26c14-1b3a-44e9-8190-d80369b87c8b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:51.222861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f26c14-1b3a-44e9-8190-d80369b87c8b None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:51.224223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:15:51.228907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e6ac-3020-4434-9d97-cea3358278e9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:51.229358 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e6ac-3020-4434-9d97-cea3358278e9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:51.229457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e6ac-3020-4434-9d97-cea3358278e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:51.248178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-bdf7e6ac-3020-4434-9d97-cea3358278e9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:51.249360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bdf7e6ac-3020-4434-9d97-cea3358278e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:51.258308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:51.412344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f26c14-1b3a-44e9-8190-d80369b87c8b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:51.427330 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f26c14-1b3a-44e9-8190-d80369b87c8b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:51.432528 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2c22-db13-4d41-bd41-b70edb95001f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:51.433160 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2c22-db13-4d41-bd41-b70edb95001f service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:51.433438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2c22-db13-4d41-bd41-b70edb95001f service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:51.433707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2c22-db13-4d41-bd41-b70edb95001f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:51.442722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2c22-db13-4d41-bd41-b70edb95001f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:51.444296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2c22-db13-4d41-bd41-b70edb95001f service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:51.453631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 762/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:55.635340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14554d8d-28f6-427b-847e-d8bc969dbd63 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:55.635873 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14554d8d-28f6-427b-847e-d8bc969dbd63 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:55.635873 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14554d8d-28f6-427b-847e-d8bc969dbd63 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:55.637180 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:15:55.642379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86733195-06e3-4188-99e8-5a58813ffb15 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:55.642825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86733195-06e3-4188-99e8-5a58813ffb15 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:55.642934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86733195-06e3-4188-99e8-5a58813ffb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:55.661877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-86733195-06e3-4188-99e8-5a58813ffb15 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:55.663056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-86733195-06e3-4188-99e8-5a58813ffb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:55.670925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:15:55.707956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14554d8d-28f6-427b-847e-d8bc969dbd63 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:55.709609 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-14554d8d-28f6-427b-847e-d8bc969dbd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:55.765770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14554d8d-28f6-427b-847e-d8bc969dbd63 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:55.769485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7edaa6-ca6e-4c5d-b424-2031000fa0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:55.770144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7edaa6-ca6e-4c5d-b424-2031000fa0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:55.770424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7edaa6-ca6e-4c5d-b424-2031000fa0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:55.770695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7edaa6-ca6e-4c5d-b424-2031000fa0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:55.779371 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7edaa6-ca6e-4c5d-b424-2031000fa0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:55.780810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7edaa6-ca6e-4c5d-b424-2031000fa0e6 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:55.785861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:55.790819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 767/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:58.000363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93c64e-53ff-4d44-91a9-c9c5e8e11ba3 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:58.000935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93c64e-53ff-4d44-91a9-c9c5e8e11ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:58.000935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93c64e-53ff-4d44-91a9-c9c5e8e11ba3 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:58.002357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:15:58.008122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e054112-e962-4571-8f49-37b6a1bee02f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:58.008580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e054112-e962-4571-8f49-37b6a1bee02f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:58.008690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e054112-e962-4571-8f49-37b6a1bee02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:58.026707 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4e054112-e962-4571-8f49-37b6a1bee02f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:58.027902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4e054112-e962-4571-8f49-37b6a1bee02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:58.035879 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:58.072436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93c64e-53ff-4d44-91a9-c9c5e8e11ba3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:58.083177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93c64e-53ff-4d44-91a9-c9c5e8e11ba3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:58.086847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63bb78-c1a6-4372-ad5c-071980e9d73c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:58.087478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63bb78-c1a6-4372-ad5c-071980e9d73c service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:58.087761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63bb78-c1a6-4372-ad5c-071980e9d73c service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:58.088034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63bb78-c1a6-4372-ad5c-071980e9d73c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:58.096881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63bb78-c1a6-4372-ad5c-071980e9d73c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:58.098376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63bb78-c1a6-4372-ad5c-071980e9d73c service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:58.107889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 765/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:01.842261 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5b6b5-1add-4229-ad87-dc17aeac58a3 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:01.842840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5b6b5-1add-4229-ad87-dc17aeac58a3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:01.842840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5b6b5-1add-4229-ad87-dc17aeac58a3 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:01.844198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:16:01.850175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145be392-59b4-4f41-8ff3-c59c6df1c84c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:01.850618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145be392-59b4-4f41-8ff3-c59c6df1c84c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:01.850717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145be392-59b4-4f41-8ff3-c59c6df1c84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:01.857005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=62 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:01.870119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-145be392-59b4-4f41-8ff3-c59c6df1c84c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:01.871305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-145be392-59b4-4f41-8ff3-c59c6df1c84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:01.879058 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:16:02.152028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5b6b5-1add-4229-ad87-dc17aeac58a3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:02.153674 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d8e5b6b5-1add-4229-ad87-dc17aeac58a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:02.212461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5b6b5-1add-4229-ad87-dc17aeac58a3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:02.215948 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ccb65-5e5e-448f-be3f-d5a4a89cf808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:02.216589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ccb65-5e5e-448f-be3f-d5a4a89cf808 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:02.216870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ccb65-5e5e-448f-be3f-d5a4a89cf808 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:02.217143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ccb65-5e5e-448f-be3f-d5a4a89cf808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:02.226046 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ccb65-5e5e-448f-be3f-d5a4a89cf808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:02.227489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ccb65-5e5e-448f-be3f-d5a4a89cf808 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:02.236631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 770/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:04.489366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418ffc5-5e7a-4430-989d-e1616d737038 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:04.489974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418ffc5-5e7a-4430-989d-e1616d737038 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:04.489974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418ffc5-5e7a-4430-989d-e1616d737038 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:04.491330 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:16:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:16:04.496766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ec7d5-2674-4ce9-ac49-47121bbf5545 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:04.497204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ec7d5-2674-4ce9-ac49-47121bbf5545 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:04.497321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ec7d5-2674-4ce9-ac49-47121bbf5545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:04.515383 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b49ec7d5-2674-4ce9-ac49-47121bbf5545 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:04.516568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b49ec7d5-2674-4ce9-ac49-47121bbf5545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:04.524718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:04.802924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418ffc5-5e7a-4430-989d-e1616d737038 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:04.804588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a418ffc5-5e7a-4430-989d-e1616d737038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:04.817791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418ffc5-5e7a-4430-989d-e1616d737038 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:04.821260 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbe384-0e18-4565-90b9-6b0a447f26b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:04.821929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbe384-0e18-4565-90b9-6b0a447f26b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:04.822216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbe384-0e18-4565-90b9-6b0a447f26b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:04.822495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbe384-0e18-4565-90b9-6b0a447f26b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:04.831290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbe384-0e18-4565-90b9-6b0a447f26b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:04.833363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbe384-0e18-4565-90b9-6b0a447f26b1 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:04.842755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 768/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:14.510113 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=32 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:14.838650 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=8 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:34.402212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dd3da-62a5-4098-bfea-3121c3bc434e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.402832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dd3da-62a5-4098-bfea-3121c3bc434e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.402832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dd3da-62a5-4098-bfea-3121c3bc434e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.438905 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-125dd3da-62a5-4098-bfea-3121c3bc434e None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:34.440919 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-125dd3da-62a5-4098-bfea-3121c3bc434e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:34.452328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 772/1540] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:34.463820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f88ecf-0735-43d3-abe7-a8a3ec534369 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.464436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f88ecf-0735-43d3-abe7-a8a3ec534369 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.464436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f88ecf-0735-43d3-abe7-a8a3ec534369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.493398 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-26f88ecf-0735-43d3-abe7-a8a3ec534369 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:34.494690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-26f88ecf-0735-43d3-abe7-a8a3ec534369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:34.513024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 769/1541] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:34.520049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125dd3da-62a5-4098-bfea-3121c3bc434e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:34.536920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125dd3da-62a5-4098-bfea-3121c3bc434e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:34.546499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fd647-f865-49dd-a6e5-def3515a44d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:34.547185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fd647-f865-49dd-a6e5-def3515a44d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.547465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fd647-f865-49dd-a6e5-def3515a44d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.547737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fd647-f865-49dd-a6e5-def3515a44d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.550084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fd647-f865-49dd-a6e5-def3515a44d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:17:34.550570 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fd647-f865-49dd-a6e5-def3515a44d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:34.552355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fd647-f865-49dd-a6e5-def3515a44d7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:34.567691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 773/1542] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:17:34.576098 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f88ecf-0735-43d3-abe7-a8a3ec534369 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:34.591806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f88ecf-0735-43d3-abe7-a8a3ec534369 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:34.597528 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc050f-dc15-4ed3-8fe9-a8557ba94092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:34.599235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc050f-dc15-4ed3-8fe9-a8557ba94092 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.599526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc050f-dc15-4ed3-8fe9-a8557ba94092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.599807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc050f-dc15-4ed3-8fe9-a8557ba94092 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.600525 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc050f-dc15-4ed3-8fe9-a8557ba94092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:34.602643 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc050f-dc15-4ed3-8fe9-a8557ba94092 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:34.658320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 770/1543] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/projects => generated 379 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:34.663260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fd647-f865-49dd-a6e5-def3515a44d7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:34.676344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fd647-f865-49dd-a6e5-def3515a44d7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:34.691964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8786419-1d30-478f-bdd3-1f19ac252019 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.692305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8786419-1d30-478f-bdd3-1f19ac252019 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.692632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8786419-1d30-478f-bdd3-1f19ac252019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.720502 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f8786419-1d30-478f-bdd3-1f19ac252019 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:34.726932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f8786419-1d30-478f-bdd3-1f19ac252019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:34.775367 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54065612-e54a-4057-8cb2-ca9106e92ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:34.776049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54065612-e54a-4057-8cb2-ca9106e92ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.776331 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54065612-e54a-4057-8cb2-ca9106e92ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.776739 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54065612-e54a-4057-8cb2-ca9106e92ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.777549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54065612-e54a-4057-8cb2-ca9106e92ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:34.779578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54065612-e54a-4057-8cb2-ca9106e92ab1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:34.821191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 771/1544] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:17:34.823599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36f1e8-f8aa-4172-b0a3-accd55aab12b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.823878 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36f1e8-f8aa-4172-b0a3-accd55aab12b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.824167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36f1e8-f8aa-4172-b0a3-accd55aab12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.828438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 774/1545] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:34.829534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69813559-12ab-4ec2-97a6-0d53b3c35ff5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.829853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69813559-12ab-4ec2-97a6-0d53b3c35ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.830133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69813559-12ab-4ec2-97a6-0d53b3c35ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.856292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-cc36f1e8-f8aa-4172-b0a3-accd55aab12b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:34.857636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cc36f1e8-f8aa-4172-b0a3-accd55aab12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:34.858893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-69813559-12ab-4ec2-97a6-0d53b3c35ff5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:34.860322 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-69813559-12ab-4ec2-97a6-0d53b3c35ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:34.866248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 772/1546] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:34.867537 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27969ea0-4cb1-417a-a6df-071b2ad980c9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.867826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27969ea0-4cb1-417a-a6df-071b2ad980c9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.868107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27969ea0-4cb1-417a-a6df-071b2ad980c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.876391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 775/1547] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:34.878864 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee19d04-f0cf-45ac-a2f5-9f5107ab8204 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.879230 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee19d04-f0cf-45ac-a2f5-9f5107ab8204 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.879514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee19d04-f0cf-45ac-a2f5-9f5107ab8204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.890069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-27969ea0-4cb1-417a-a6df-071b2ad980c9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:34.891273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-27969ea0-4cb1-417a-a6df-071b2ad980c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:34.906091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 773/1548] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:34.906942 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27969ea0-4cb1-417a-a6df-071b2ad980c9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:34.915819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-dee19d04-f0cf-45ac-a2f5-9f5107ab8204 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:34.917037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-dee19d04-f0cf-45ac-a2f5-9f5107ab8204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:34.918145 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27969ea0-4cb1-417a-a6df-071b2ad980c9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:34.923442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed42ee6-b3e0-4698-8890-7ff02707c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:34.924309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed42ee6-b3e0-4698-8890-7ff02707c4ea admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.924679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed42ee6-b3e0-4698-8890-7ff02707c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.925197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed42ee6-b3e0-4698-8890-7ff02707c4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.926245 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed42ee6-b3e0-4698-8890-7ff02707c4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:34.927873 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed42ee6-b3e0-4698-8890-7ff02707c4ea admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:34.929965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 776/1549] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:34.931419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0419599-e756-4a3d-8217-f959575b796f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.931713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0419599-e756-4a3d-8217-f959575b796f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.932109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0419599-e756-4a3d-8217-f959575b796f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.940402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 774/1550] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:34.941896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a07b1-94d0-462b-8129-5dde5d00c102 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.942194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a07b1-94d0-462b-8129-5dde5d00c102 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.942473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a07b1-94d0-462b-8129-5dde5d00c102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.954037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d0419599-e756-4a3d-8217-f959575b796f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:34.955248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d0419599-e756-4a3d-8217-f959575b796f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:34.962737 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-825a07b1-94d0-462b-8129-5dde5d00c102 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:34.963927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-825a07b1-94d0-462b-8129-5dde5d00c102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:34.965847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 777/1551] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:34.970118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7d281-db11-46a9-8406-9f11e1b9823c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.970414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7d281-db11-46a9-8406-9f11e1b9823c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.970881 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7d281-db11-46a9-8406-9f11e1b9823c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.975410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 775/1552] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:34.977072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e5ccc-9afa-44f5-a6f0-2d7f5fd0cd39 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:34.977509 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e5ccc-9afa-44f5-a6f0-2d7f5fd0cd39 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:34.977875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e5ccc-9afa-44f5-a6f0-2d7f5fd0cd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:34.991790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-22f7d281-db11-46a9-8406-9f11e1b9823c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:34.992986 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-22f7d281-db11-46a9-8406-9f11e1b9823c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:34.996355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c41e5ccc-9afa-44f5-a6f0-2d7f5fd0cd39 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:34.997518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c41e5ccc-9afa-44f5-a6f0-2d7f5fd0cd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:35.000792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 778/1553] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:35.001553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7d281-db11-46a9-8406-9f11e1b9823c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.008738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 776/1554] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:35.009533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e5ccc-9afa-44f5-a6f0-2d7f5fd0cd39 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.013282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7d281-db11-46a9-8406-9f11e1b9823c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.019392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fdfb3-ae1a-4ee1-b1c0-43205f206520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.019539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fdfb3-ae1a-4ee1-b1c0-43205f206520 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.019762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fdfb3-ae1a-4ee1-b1c0-43205f206520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.020040 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fdfb3-ae1a-4ee1-b1c0-43205f206520 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.020763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264fdfb3-ae1a-4ee1-b1c0-43205f206520 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:17:35.020883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e5ccc-9afa-44f5-a6f0-2d7f5fd0cd39 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.021068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264fdfb3-ae1a-4ee1-b1c0-43205f206520 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.022720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264fdfb3-ae1a-4ee1-b1c0-43205f206520 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.025397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77f2fe-771c-4590-99c8-39c66c80f5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.026070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77f2fe-771c-4590-99c8-39c66c80f5cf admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.026355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77f2fe-771c-4590-99c8-39c66c80f5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.026631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77f2fe-771c-4590-99c8-39c66c80f5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.027345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77f2fe-771c-4590-99c8-39c66c80f5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.028913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77f2fe-771c-4590-99c8-39c66c80f5cf admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.034681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 779/1555] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:17:35.036118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17105d60-02f3-452a-9b06-94dadd46330a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.036406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17105d60-02f3-452a-9b06-94dadd46330a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.036679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17105d60-02f3-452a-9b06-94dadd46330a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.042387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 777/1556] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:35.042387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77f2fe-771c-4590-99c8-39c66c80f5cf admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.051314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77f2fe-771c-4590-99c8-39c66c80f5cf admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.055630 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-17105d60-02f3-452a-9b06-94dadd46330a None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:35.055951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e867af6-1b6e-4fa0-948c-c4b7dcc24038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.056592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e867af6-1b6e-4fa0-948c-c4b7dcc24038 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.056821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-17105d60-02f3-452a-9b06-94dadd46330a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:35.056929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e867af6-1b6e-4fa0-948c-c4b7dcc24038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.057134 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e867af6-1b6e-4fa0-948c-c4b7dcc24038 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.057876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e867af6-1b6e-4fa0-948c-c4b7dcc24038 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:17:35.058187 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e867af6-1b6e-4fa0-948c-c4b7dcc24038 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.059200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-3e867af6-1b6e-4fa0-948c-c4b7dcc24038 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 18:17:35.059200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: warnings.warn(msg) May 26 18:17:35.059200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: May 26 18:17:35.060069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e867af6-1b6e-4fa0-948c-c4b7dcc24038 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.064907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 780/1557] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:35.066830 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6ca1e-2d8a-42ab-8c0c-3bd0cd5e2c6f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.067122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6ca1e-2d8a-42ab-8c0c-3bd0cd5e2c6f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.067398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6ca1e-2d8a-42ab-8c0c-3bd0cd5e2c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.086386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 778/1558] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:17:35.086386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318407f-daa1-4f69-920e-02c7741db35d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.086386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318407f-daa1-4f69-920e-02c7741db35d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.086386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318407f-daa1-4f69-920e-02c7741db35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.092984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f1d6ca1e-2d8a-42ab-8c0c-3bd0cd5e2c6f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:35.094272 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f1d6ca1e-2d8a-42ab-8c0c-3bd0cd5e2c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:35.098041 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-6318407f-daa1-4f69-920e-02c7741db35d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:35.099220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6318407f-daa1-4f69-920e-02c7741db35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:35.102083 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 781/1559] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:35.102979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6ca1e-2d8a-42ab-8c0c-3bd0cd5e2c6f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.112698 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6ca1e-2d8a-42ab-8c0c-3bd0cd5e2c6f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.114968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 779/1560] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:35.115788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318407f-daa1-4f69-920e-02c7741db35d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.117479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db64d8-0f82-4aec-8992-f08bb5cdec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.118165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db64d8-0f82-4aec-8992-f08bb5cdec24 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.118576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db64d8-0f82-4aec-8992-f08bb5cdec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.118802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db64d8-0f82-4aec-8992-f08bb5cdec24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.119528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db64d8-0f82-4aec-8992-f08bb5cdec24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:17:35.119835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db64d8-0f82-4aec-8992-f08bb5cdec24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.121448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db64d8-0f82-4aec-8992-f08bb5cdec24 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.124950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318407f-daa1-4f69-920e-02c7741db35d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.130159 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6720ff-be86-4847-ae0d-8b2ba371978f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.130794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6720ff-be86-4847-ae0d-8b2ba371978f admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.131075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6720ff-be86-4847-ae0d-8b2ba371978f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.131347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6720ff-be86-4847-ae0d-8b2ba371978f admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.132063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6720ff-be86-4847-ae0d-8b2ba371978f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:17:35.132363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6720ff-be86-4847-ae0d-8b2ba371978f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.134129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6720ff-be86-4847-ae0d-8b2ba371978f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.134266 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 782/1561] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:17:35.135033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db64d8-0f82-4aec-8992-f08bb5cdec24 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.145115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db64d8-0f82-4aec-8992-f08bb5cdec24 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.153052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 780/1562] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:17:35.153052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6720ff-be86-4847-ae0d-8b2ba371978f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.156721 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99904472-6e57-4ace-9cef-801941b4b051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.157384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99904472-6e57-4ace-9cef-801941b4b051 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.157669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99904472-6e57-4ace-9cef-801941b4b051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.157934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6720ff-be86-4847-ae0d-8b2ba371978f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.158033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99904472-6e57-4ace-9cef-801941b4b051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.158999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99904472-6e57-4ace-9cef-801941b4b051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.160042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-99904472-6e57-4ace-9cef-801941b4b051 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:17:35.160042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: warnings.warn(msg) May 26 18:17:35.160042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: May 26 18:17:35.161381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99904472-6e57-4ace-9cef-801941b4b051 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.162581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2005a-e042-4a13-854e-227863fd3ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.163430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2005a-e042-4a13-854e-227863fd3ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.163708 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2005a-e042-4a13-854e-227863fd3ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.163982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2005a-e042-4a13-854e-227863fd3ee1 admin admin] PATH_INFO: `/v3/projects/3f65ddb707cf4a3991b4797726d1336e/users/02438387c012405eb258a65311c092a5/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.179090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2005a-e042-4a13-854e-227863fd3ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f65ddb707cf4a3991b4797726d1336e, user_id=02438387c012405eb258a65311c092a5, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.181722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2005a-e042-4a13-854e-227863fd3ee1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.190372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 783/1563] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:35.192099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df596ddf-3ec4-4f61-99ea-afd50921f3d8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.192392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df596ddf-3ec4-4f61-99ea-afd50921f3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.192670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df596ddf-3ec4-4f61-99ea-afd50921f3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.197980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 781/1564] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:35 2022] PUT /identity/v3/projects/3f65ddb707cf4a3991b4797726d1336e/users/02438387c012405eb258a65311c092a5/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:35.198773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2005a-e042-4a13-854e-227863fd3ee1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.211089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-df596ddf-3ec4-4f61-99ea-afd50921f3d8 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:35.212574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-df596ddf-3ec4-4f61-99ea-afd50921f3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:35.231798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:35.255499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:35.271074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2005a-e042-4a13-854e-227863fd3ee1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.275692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c859ee-46ec-4017-812b-28f42b229bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.276307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c859ee-46ec-4017-812b-28f42b229bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.276584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c859ee-46ec-4017-812b-28f42b229bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.276859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c859ee-46ec-4017-812b-28f42b229bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.277572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c859ee-46ec-4017-812b-28f42b229bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.278043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 784/1565] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:35.279320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874145be-0cce-4f8e-8d9b-0d99be2646ba None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.279508 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c859ee-46ec-4017-812b-28f42b229bb4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.279620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874145be-0cce-4f8e-8d9b-0d99be2646ba None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.279857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874145be-0cce-4f8e-8d9b-0d99be2646ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.298144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-874145be-0cce-4f8e-8d9b-0d99be2646ba None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:35.299508 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-874145be-0cce-4f8e-8d9b-0d99be2646ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:35.309316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 782/1566] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:35.309316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c859ee-46ec-4017-812b-28f42b229bb4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.309566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 785/1567] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:35.310572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874145be-0cce-4f8e-8d9b-0d99be2646ba None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.322522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874145be-0cce-4f8e-8d9b-0d99be2646ba None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.373366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c859ee-46ec-4017-812b-28f42b229bb4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.378166 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083bfe6-3bcd-4547-929f-871791e5d831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.378835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083bfe6-3bcd-4547-929f-871791e5d831 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.379143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083bfe6-3bcd-4547-929f-871791e5d831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.379410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083bfe6-3bcd-4547-929f-871791e5d831 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.379518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29f8fd-6e99-426e-9619-980b46a4cab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.380077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29f8fd-6e99-426e-9619-980b46a4cab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.380190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083bfe6-3bcd-4547-929f-871791e5d831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:17:35.380341 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29f8fd-6e99-426e-9619-980b46a4cab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.380445 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083bfe6-3bcd-4547-929f-871791e5d831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.380605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29f8fd-6e99-426e-9619-980b46a4cab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.381411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29f8fd-6e99-426e-9619-980b46a4cab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.382061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083bfe6-3bcd-4547-929f-871791e5d831 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.383485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29f8fd-6e99-426e-9619-980b46a4cab9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.395090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 783/1568] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:17:35.395875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083bfe6-3bcd-4547-929f-871791e5d831 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.408108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083bfe6-3bcd-4547-929f-871791e5d831 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.411650 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 786/1569] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:35.412530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29f8fd-6e99-426e-9619-980b46a4cab9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.415993 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df04fba-e5ce-454b-bcba-f4cafc3c8f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.416646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df04fba-e5ce-454b-bcba-f4cafc3c8f98 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.416923 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df04fba-e5ce-454b-bcba-f4cafc3c8f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.417189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df04fba-e5ce-454b-bcba-f4cafc3c8f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.417910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df04fba-e5ce-454b-bcba-f4cafc3c8f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.419853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df04fba-e5ce-454b-bcba-f4cafc3c8f98 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.449202 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 784/1570] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:35.450849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b057f2-baeb-4ff9-9261-5ef667ffc3cd None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.451240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b057f2-baeb-4ff9-9261-5ef667ffc3cd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.451567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b057f2-baeb-4ff9-9261-5ef667ffc3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.470146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d2b057f2-baeb-4ff9-9261-5ef667ffc3cd None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:35.471775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d2b057f2-baeb-4ff9-9261-5ef667ffc3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:35.478504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29f8fd-6e99-426e-9619-980b46a4cab9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.532084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 785/1571] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:35.532902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b057f2-baeb-4ff9-9261-5ef667ffc3cd None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.536430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6da4e-aba7-4c9b-88ef-f5d48a0fa03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.536768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6da4e-aba7-4c9b-88ef-f5d48a0fa03f admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.537051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6da4e-aba7-4c9b-88ef-f5d48a0fa03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.537332 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6da4e-aba7-4c9b-88ef-f5d48a0fa03f admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.538059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6da4e-aba7-4c9b-88ef-f5d48a0fa03f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.540000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6da4e-aba7-4c9b-88ef-f5d48a0fa03f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.542630 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b057f2-baeb-4ff9-9261-5ef667ffc3cd None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.547546 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac44ce-efd1-4345-8e6e-38e50cf3a502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.548196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac44ce-efd1-4345-8e6e-38e50cf3a502 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.548474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac44ce-efd1-4345-8e6e-38e50cf3a502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.548745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac44ce-efd1-4345-8e6e-38e50cf3a502 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.549496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac44ce-efd1-4345-8e6e-38e50cf3a502 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:17:35.549948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac44ce-efd1-4345-8e6e-38e50cf3a502 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.551824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac44ce-efd1-4345-8e6e-38e50cf3a502 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.557140 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:35.564568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 786/1572] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:17:35.566150 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac44ce-efd1-4345-8e6e-38e50cf3a502 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.575720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 787/1573] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:35.575720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6da4e-aba7-4c9b-88ef-f5d48a0fa03f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.576119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac44ce-efd1-4345-8e6e-38e50cf3a502 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.581297 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0b8e6-7b95-4af9-9743-fcb1f0ed2622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.582361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0b8e6-7b95-4af9-9743-fcb1f0ed2622 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.582692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0b8e6-7b95-4af9-9743-fcb1f0ed2622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.583000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0b8e6-7b95-4af9-9743-fcb1f0ed2622 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.583817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0b8e6-7b95-4af9-9743-fcb1f0ed2622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.584858 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-8bb0b8e6-7b95-4af9-9743-fcb1f0ed2622 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:17:35.584858 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: warnings.warn(msg) May 26 18:17:35.584858 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: May 26 18:17:35.586186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0b8e6-7b95-4af9-9743-fcb1f0ed2622 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.586789 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6da4e-aba7-4c9b-88ef-f5d48a0fa03f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.591328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3c836-8d6e-4dc3-ab9a-97097993041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.591982 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3c836-8d6e-4dc3-ab9a-97097993041c admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.592292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3c836-8d6e-4dc3-ab9a-97097993041c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.592571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3c836-8d6e-4dc3-ab9a-97097993041c admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.593328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3c836-8d6e-4dc3-ab9a-97097993041c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:17:35.593626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3c836-8d6e-4dc3-ab9a-97097993041c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.595187 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:35.595316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3c836-8d6e-4dc3-ab9a-97097993041c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.611378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 787/1574] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/users => generated 288 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:35.611378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0b8e6-7b95-4af9-9743-fcb1f0ed2622 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.611798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 788/1575] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:17:35.611798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3c836-8d6e-4dc3-ab9a-97097993041c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.626265 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3c836-8d6e-4dc3-ab9a-97097993041c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.626439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0b8e6-7b95-4af9-9743-fcb1f0ed2622 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.630968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11849a78-27ea-4feb-a0cb-8ad406eb8266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.631226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4e5c5-b72f-443c-a30f-a185b74f9397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.631622 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11849a78-27ea-4feb-a0cb-8ad406eb8266 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.631906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11849a78-27ea-4feb-a0cb-8ad406eb8266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.632005 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4e5c5-b72f-443c-a30f-a185b74f9397 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.632181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11849a78-27ea-4feb-a0cb-8ad406eb8266 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.632294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4e5c5-b72f-443c-a30f-a185b74f9397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.632450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4e5c5-b72f-443c-a30f-a185b74f9397 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.632870 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11849a78-27ea-4feb-a0cb-8ad406eb8266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.633123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4e5c5-b72f-443c-a30f-a185b74f9397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.634918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11849a78-27ea-4feb-a0cb-8ad406eb8266 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.635213 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4e5c5-b72f-443c-a30f-a185b74f9397 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.659699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 789/1576] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/users => generated 283 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:35.660806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4e5c5-b72f-443c-a30f-a185b74f9397 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.669776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 788/1577] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/projects => generated 405 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:35.670847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11849a78-27ea-4feb-a0cb-8ad406eb8266 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.671035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4e5c5-b72f-443c-a30f-a185b74f9397 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.726530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11849a78-27ea-4feb-a0cb-8ad406eb8266 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.728987 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a99ed-dcbc-46e6-9233-ae6c97278f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.729632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a99ed-dcbc-46e6-9233-ae6c97278f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.729943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a99ed-dcbc-46e6-9233-ae6c97278f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.730220 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a99ed-dcbc-46e6-9233-ae6c97278f1c admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.730400 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9faa905-adde-4260-beca-1bd39360a428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.730915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a99ed-dcbc-46e6-9233-ae6c97278f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.731044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9faa905-adde-4260-beca-1bd39360a428 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.731327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9faa905-adde-4260-beca-1bd39360a428 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.731632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9faa905-adde-4260-beca-1bd39360a428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.732890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a99ed-dcbc-46e6-9233-ae6c97278f1c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.741898 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9faa905-adde-4260-beca-1bd39360a428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.743329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9faa905-adde-4260-beca-1bd39360a428 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.754192 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 790/1578] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:35.754992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a99ed-dcbc-46e6-9233-ae6c97278f1c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.757322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 789/1579] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:35.758189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9faa905-adde-4260-beca-1bd39360a428 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.764280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a99ed-dcbc-46e6-9233-ae6c97278f1c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.767427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9faa905-adde-4260-beca-1bd39360a428 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.770309 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815ddd1-7468-45e6-9af3-0b30afa75fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.771205 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815ddd1-7468-45e6-9af3-0b30afa75fca admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.771489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815ddd1-7468-45e6-9af3-0b30afa75fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.771763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815ddd1-7468-45e6-9af3-0b30afa75fca admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.772517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815ddd1-7468-45e6-9af3-0b30afa75fca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.773108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b1777-ffdd-4012-a351-afb03caff282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.773879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b1777-ffdd-4012-a351-afb03caff282 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.774229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b1777-ffdd-4012-a351-afb03caff282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.774408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815ddd1-7468-45e6-9af3-0b30afa75fca admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.774636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b1777-ffdd-4012-a351-afb03caff282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.775454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b1777-ffdd-4012-a351-afb03caff282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.777034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b1777-ffdd-4012-a351-afb03caff282 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.789774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 790/1580] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:35.790024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b1777-ffdd-4012-a351-afb03caff282 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.802870 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b1777-ffdd-4012-a351-afb03caff282 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.808305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c012a97-d9b2-42bc-b4d7-b24df0384427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.809077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c012a97-d9b2-42bc-b4d7-b24df0384427 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.809389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c012a97-d9b2-42bc-b4d7-b24df0384427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.809670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c012a97-d9b2-42bc-b4d7-b24df0384427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.810419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c012a97-d9b2-42bc-b4d7-b24df0384427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.814346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 791/1581] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:35.814346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815ddd1-7468-45e6-9af3-0b30afa75fca admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.814600 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c012a97-d9b2-42bc-b4d7-b24df0384427 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.825660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 791/1582] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:35.826708 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c012a97-d9b2-42bc-b4d7-b24df0384427 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.838618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c012a97-d9b2-42bc-b4d7-b24df0384427 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.889332 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815ddd1-7468-45e6-9af3-0b30afa75fca admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.895448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b8be2-a347-40b3-aeb2-ad56221e94c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.896226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b8be2-a347-40b3-aeb2-ad56221e94c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.896568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b8be2-a347-40b3-aeb2-ad56221e94c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.896899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b8be2-a347-40b3-aeb2-ad56221e94c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.897676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b8be2-a347-40b3-aeb2-ad56221e94c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.900093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b8be2-a347-40b3-aeb2-ad56221e94c7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.912611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524b09a-bdf2-4a5b-bf46-d14e571d8a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:35.913203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524b09a-bdf2-4a5b-bf46-d14e571d8a57 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:35.913483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524b09a-bdf2-4a5b-bf46-d14e571d8a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:35.913783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524b09a-bdf2-4a5b-bf46-d14e571d8a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:35.914568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524b09a-bdf2-4a5b-bf46-d14e571d8a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:35.916220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524b09a-bdf2-4a5b-bf46-d14e571d8a57 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:35.929178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 792/1583] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:35.929916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524b09a-bdf2-4a5b-bf46-d14e571d8a57 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.937490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 792/1584] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:35.937490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b8be2-a347-40b3-aeb2-ad56221e94c7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:35.940552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524b09a-bdf2-4a5b-bf46-d14e571d8a57 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:35.962973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:36.005802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:36.021152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b8be2-a347-40b3-aeb2-ad56221e94c7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.021368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40055858-cd37-48b6-8c6f-6f5970645223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.022072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40055858-cd37-48b6-8c6f-6f5970645223 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.022427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40055858-cd37-48b6-8c6f-6f5970645223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.022879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40055858-cd37-48b6-8c6f-6f5970645223 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.023663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40055858-cd37-48b6-8c6f-6f5970645223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.025689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40055858-cd37-48b6-8c6f-6f5970645223 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.026781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3217d81-a60a-4f05-b52b-727db64ad4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.027574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3217d81-a60a-4f05-b52b-727db64ad4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.027989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3217d81-a60a-4f05-b52b-727db64ad4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.028346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3217d81-a60a-4f05-b52b-727db64ad4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.029222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3217d81-a60a-4f05-b52b-727db64ad4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.031030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3217d81-a60a-4f05-b52b-727db64ad4b4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.042485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 793/1585] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:36.043653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3217d81-a60a-4f05-b52b-727db64ad4b4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.052232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 793/1586] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:35 2022] POST /identity/v3/users => generated 303 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:36.052232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40055858-cd37-48b6-8c6f-6f5970645223 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.057807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3217d81-a60a-4f05-b52b-727db64ad4b4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.061403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40055858-cd37-48b6-8c6f-6f5970645223 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.062643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1829e-460e-4d50-8499-0b1cd193dc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.063465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1829e-460e-4d50-8499-0b1cd193dc8b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.070342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1829e-460e-4d50-8499-0b1cd193dc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.070342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1829e-460e-4d50-8499-0b1cd193dc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.070342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1829e-460e-4d50-8499-0b1cd193dc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.070342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1829e-460e-4d50-8499-0b1cd193dc8b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.072278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515270a6-7c58-46da-87e8-8efedc6ed72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.072956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515270a6-7c58-46da-87e8-8efedc6ed72e admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.073230 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515270a6-7c58-46da-87e8-8efedc6ed72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.073495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515270a6-7c58-46da-87e8-8efedc6ed72e admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.074399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515270a6-7c58-46da-87e8-8efedc6ed72e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.076461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515270a6-7c58-46da-87e8-8efedc6ed72e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.083253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 794/1587] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:36.084803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1829e-460e-4d50-8499-0b1cd193dc8b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.098481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1829e-460e-4d50-8499-0b1cd193dc8b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.104021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 794/1588] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:36 2022] POST /identity/v3/users => generated 296 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:36.104021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515270a6-7c58-46da-87e8-8efedc6ed72e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.106470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241667d-de6e-4fcd-a08d-8aee12ec4d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.107399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241667d-de6e-4fcd-a08d-8aee12ec4d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.107797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241667d-de6e-4fcd-a08d-8aee12ec4d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.108489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241667d-de6e-4fcd-a08d-8aee12ec4d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.109179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241667d-de6e-4fcd-a08d-8aee12ec4d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.111182 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241667d-de6e-4fcd-a08d-8aee12ec4d0c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.113813 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515270a6-7c58-46da-87e8-8efedc6ed72e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.121381 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea703d-7e91-445d-b3b8-5e91a7a427bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.122251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea703d-7e91-445d-b3b8-5e91a7a427bb admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.122528 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea703d-7e91-445d-b3b8-5e91a7a427bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.122817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea703d-7e91-445d-b3b8-5e91a7a427bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.126776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 795/1589] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:36.128479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241667d-de6e-4fcd-a08d-8aee12ec4d0c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.129234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea703d-7e91-445d-b3b8-5e91a7a427bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.131063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea703d-7e91-445d-b3b8-5e91a7a427bb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.148550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241667d-de6e-4fcd-a08d-8aee12ec4d0c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.150337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 795/1590] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:36.152100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea703d-7e91-445d-b3b8-5e91a7a427bb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.157506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319a4ce-c0ca-43d1-9425-23569d1cf1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.158233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319a4ce-c0ca-43d1-9425-23569d1cf1fb admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.159699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319a4ce-c0ca-43d1-9425-23569d1cf1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.159986 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319a4ce-c0ca-43d1-9425-23569d1cf1fb admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.165200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319a4ce-c0ca-43d1-9425-23569d1cf1fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.166261 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea703d-7e91-445d-b3b8-5e91a7a427bb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.169450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319a4ce-c0ca-43d1-9425-23569d1cf1fb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.172117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e84c5a-e8b9-410e-9acb-db81703711b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.174718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e84c5a-e8b9-410e-9acb-db81703711b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.175308 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e84c5a-e8b9-410e-9acb-db81703711b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.175735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e84c5a-e8b9-410e-9acb-db81703711b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.176717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e84c5a-e8b9-410e-9acb-db81703711b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.179567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e84c5a-e8b9-410e-9acb-db81703711b3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.195274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 796/1591] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:36.195274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e84c5a-e8b9-410e-9acb-db81703711b3 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.201930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 796/1592] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:36 2022] POST /identity/v3/users => generated 303 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:36.202816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319a4ce-c0ca-43d1-9425-23569d1cf1fb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.210403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e84c5a-e8b9-410e-9acb-db81703711b3 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.213505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319a4ce-c0ca-43d1-9425-23569d1cf1fb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.216741 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec169d-d438-4b58-856a-00e8eb4f93b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.217407 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec169d-d438-4b58-856a-00e8eb4f93b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.217826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec169d-d438-4b58-856a-00e8eb4f93b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.218109 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec169d-d438-4b58-856a-00e8eb4f93b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.218331 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f4ac9-089e-4e24-bb84-5c7beabdb3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.218826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec169d-d438-4b58-856a-00e8eb4f93b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.219339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f4ac9-089e-4e24-bb84-5c7beabdb3fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.219724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f4ac9-089e-4e24-bb84-5c7beabdb3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.220068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f4ac9-089e-4e24-bb84-5c7beabdb3fe admin admin] PATH_INFO: `/v3/projects/0c2a09a6e81240889c49b5abb24ab225/users/a20f6932d763461ea26c2d60ed58a59f/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.220427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec169d-d438-4b58-856a-00e8eb4f93b7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.233776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 797/1593] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:36.233976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec169d-d438-4b58-856a-00e8eb4f93b7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.239564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f4ac9-089e-4e24-bb84-5c7beabdb3fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c2a09a6e81240889c49b5abb24ab225, user_id=a20f6932d763461ea26c2d60ed58a59f, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.240999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-b95f4ac9-089e-4e24-bb84-5c7beabdb3fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:17:36.240999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: warnings.warn(msg) May 26 18:17:36.240999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: May 26 18:17:36.243274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f4ac9-089e-4e24-bb84-5c7beabdb3fe admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.249572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec169d-d438-4b58-856a-00e8eb4f93b7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.255827 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb60b3-84c6-4a6f-bc3f-b12a61e3ba4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.256497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb60b3-84c6-4a6f-bc3f-b12a61e3ba4f admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.257208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb60b3-84c6-4a6f-bc3f-b12a61e3ba4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.257208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb60b3-84c6-4a6f-bc3f-b12a61e3ba4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.257935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb60b3-84c6-4a6f-bc3f-b12a61e3ba4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.259706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb60b3-84c6-4a6f-bc3f-b12a61e3ba4f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.267340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 797/1594] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:36 2022] PUT /identity/v3/projects/0c2a09a6e81240889c49b5abb24ab225/users/a20f6932d763461ea26c2d60ed58a59f/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:36.268411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f4ac9-089e-4e24-bb84-5c7beabdb3fe admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.274887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 798/1595] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:36.275917 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb60b3-84c6-4a6f-bc3f-b12a61e3ba4f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.289405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb60b3-84c6-4a6f-bc3f-b12a61e3ba4f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.348331 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f4ac9-089e-4e24-bb84-5c7beabdb3fe admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.354061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66b78a-4074-4bae-97fb-76d79cb2e6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.355241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66b78a-4074-4bae-97fb-76d79cb2e6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.355617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66b78a-4074-4bae-97fb-76d79cb2e6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.355895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66b78a-4074-4bae-97fb-76d79cb2e6c6 admin admin] PATH_INFO: `/v3/projects/ab2f116ff48f4fa38a88217603c4500a/users/e73264257ac541269c1685b5355d4d1c/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.358636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597f106-f92e-4cd2-a2dc-0b3eab95bc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.359542 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597f106-f92e-4cd2-a2dc-0b3eab95bc20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.359826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597f106-f92e-4cd2-a2dc-0b3eab95bc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.360100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597f106-f92e-4cd2-a2dc-0b3eab95bc20 admin admin] PATH_INFO: `/v3/projects/b6d18f4b036f492da77cb37f19c75827/users/b9de367933b64fb0833d7ae095e02c7f/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.374851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66b78a-4074-4bae-97fb-76d79cb2e6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab2f116ff48f4fa38a88217603c4500a, user_id=e73264257ac541269c1685b5355d4d1c, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.377442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597f106-f92e-4cd2-a2dc-0b3eab95bc20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d18f4b036f492da77cb37f19c75827, user_id=b9de367933b64fb0833d7ae095e02c7f, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.378072 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66b78a-4074-4bae-97fb-76d79cb2e6c6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.380905 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597f106-f92e-4cd2-a2dc-0b3eab95bc20 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.394549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 798/1596] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:36 2022] PUT /identity/v3/projects/ab2f116ff48f4fa38a88217603c4500a/users/e73264257ac541269c1685b5355d4d1c/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:36.395543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66b78a-4074-4bae-97fb-76d79cb2e6c6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.396735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 799/1597] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:36 2022] PUT /identity/v3/projects/b6d18f4b036f492da77cb37f19c75827/users/b9de367933b64fb0833d7ae095e02c7f/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:36.406333 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597f106-f92e-4cd2-a2dc-0b3eab95bc20 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.451218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:36.479832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66b78a-4074-4bae-97fb-76d79cb2e6c6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.482408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597f106-f92e-4cd2-a2dc-0b3eab95bc20 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.484671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ed492-af7f-4ef8-9ebe-2f96d1e37055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.485611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ed492-af7f-4ef8-9ebe-2f96d1e37055 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.485884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ed492-af7f-4ef8-9ebe-2f96d1e37055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.486184 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ed492-af7f-4ef8-9ebe-2f96d1e37055 admin admin] PATH_INFO: `/v3/projects/31b4afa4107c4cd2ac0212c45d998801/users/8da0a5a09a6041d68c5e445af1a492f3/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.489774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20879af5-7e27-4e7a-8b54-d2669d32ca61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.490452 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20879af5-7e27-4e7a-8b54-d2669d32ca61 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.491161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20879af5-7e27-4e7a-8b54-d2669d32ca61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.491161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20879af5-7e27-4e7a-8b54-d2669d32ca61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.491771 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20879af5-7e27-4e7a-8b54-d2669d32ca61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.494222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20879af5-7e27-4e7a-8b54-d2669d32ca61 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.496433 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:36.504245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ed492-af7f-4ef8-9ebe-2f96d1e37055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b4afa4107c4cd2ac0212c45d998801, user_id=8da0a5a09a6041d68c5e445af1a492f3, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.507519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ed492-af7f-4ef8-9ebe-2f96d1e37055 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.511222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 800/1598] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:36.511222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20879af5-7e27-4e7a-8b54-d2669d32ca61 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.521966 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20879af5-7e27-4e7a-8b54-d2669d32ca61 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.528718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 799/1599] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:36 2022] PUT /identity/v3/projects/31b4afa4107c4cd2ac0212c45d998801/users/8da0a5a09a6041d68c5e445af1a492f3/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:36.536033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb710bb-c52c-42ef-9902-7bfc92da736c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.536033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb710bb-c52c-42ef-9902-7bfc92da736c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.536190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb710bb-c52c-42ef-9902-7bfc92da736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.558271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-feb710bb-c52c-42ef-9902-7bfc92da736c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:36.559557 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-feb710bb-c52c-42ef-9902-7bfc92da736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:36.599471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2fd516-eed5-4549-99aa-5b978ed15232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.600481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fd516-eed5-4549-99aa-5b978ed15232 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.600706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fd516-eed5-4549-99aa-5b978ed15232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.600999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fd516-eed5-4549-99aa-5b978ed15232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.601745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2fd516-eed5-4549-99aa-5b978ed15232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.603398 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2fd516-eed5-4549-99aa-5b978ed15232 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.615124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 801/1600] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:36.615977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2fd516-eed5-4549-99aa-5b978ed15232 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.628168 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2fd516-eed5-4549-99aa-5b978ed15232 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.632991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767050ea-90e9-40ca-aa7d-b8f250df0a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.633654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767050ea-90e9-40ca-aa7d-b8f250df0a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.633968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767050ea-90e9-40ca-aa7d-b8f250df0a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.634247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767050ea-90e9-40ca-aa7d-b8f250df0a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.635034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 800/1601] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:36.635163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767050ea-90e9-40ca-aa7d-b8f250df0a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.636539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767050ea-90e9-40ca-aa7d-b8f250df0a8c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.643983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a99cea-2b38-4747-b8a6-3304e5c32e89 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.644482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a99cea-2b38-4747-b8a6-3304e5c32e89 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.644990 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a99cea-2b38-4747-b8a6-3304e5c32e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.654756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 802/1602] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:36.656008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab292d-ec0c-42e1-83e9-538fce87ac35 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.656293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab292d-ec0c-42e1-83e9-538fce87ac35 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.656629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab292d-ec0c-42e1-83e9-538fce87ac35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.682305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b9a99cea-2b38-4747-b8a6-3304e5c32e89 None None] MFA Rules not processed for user `e73264257ac541269c1685b5355d4d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:36.683611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b9a99cea-2b38-4747-b8a6-3304e5c32e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:36.688551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e3ab292d-ec0c-42e1-83e9-538fce87ac35 None None] MFA Rules not processed for user `b9de367933b64fb0833d7ae095e02c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:36.690854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e3ab292d-ec0c-42e1-83e9-538fce87ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:36.761882 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 801/1603] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:36.763529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a99cea-2b38-4747-b8a6-3304e5c32e89 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.770616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 803/1604] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:36.770616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6d3eb-3f0e-481b-923a-09b0a4cac6ac None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.770616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6d3eb-3f0e-481b-923a-09b0a4cac6ac None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.770616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6d3eb-3f0e-481b-923a-09b0a4cac6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.777272 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a99cea-2b38-4747-b8a6-3304e5c32e89 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.782743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d3f0e-7c5b-42b4-a428-344b0b899e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.783801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d3f0e-7c5b-42b4-a428-344b0b899e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.784276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d3f0e-7c5b-42b4-a428-344b0b899e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.784681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d3f0e-7c5b-42b4-a428-344b0b899e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.785484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d3f0e-7c5b-42b4-a428-344b0b899e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.787267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d3f0e-7c5b-42b4-a428-344b0b899e00 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.787408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-06e6d3eb-3f0e-481b-923a-09b0a4cac6ac None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:36.788693 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-06e6d3eb-3f0e-481b-923a-09b0a4cac6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:36.798732 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 804/1605] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:36.800579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 802/1606] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:36.800827 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ea421-08b5-46aa-af74-4be968520597 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.801117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ea421-08b5-46aa-af74-4be968520597 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.801531 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ea421-08b5-46aa-af74-4be968520597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.801962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d3f0e-7c5b-42b4-a428-344b0b899e00 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.817488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d3f0e-7c5b-42b4-a428-344b0b899e00 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.823083 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734cbb3-063c-4a9e-a988-73ea1bf12c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.824069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734cbb3-063c-4a9e-a988-73ea1bf12c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.824446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734cbb3-063c-4a9e-a988-73ea1bf12c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.824815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734cbb3-063c-4a9e-a988-73ea1bf12c61 admin admin] PATH_INFO: `/v3/projects/7540c93de42e46ae9c4c02ae1c20032d/users/96a0ff77014d41beb98e5748f8f255a8/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.836077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-313ea421-08b5-46aa-af74-4be968520597 None None] MFA Rules not processed for user `02438387c012405eb258a65311c092a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:36.837294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-313ea421-08b5-46aa-af74-4be968520597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:36.841199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734cbb3-063c-4a9e-a988-73ea1bf12c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7540c93de42e46ae9c4c02ae1c20032d, user_id=96a0ff77014d41beb98e5748f8f255a8, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.843961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734cbb3-063c-4a9e-a988-73ea1bf12c61 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:36.858279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 803/1607] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:36 2022] PUT /identity/v3/projects/7540c93de42e46ae9c4c02ae1c20032d/users/96a0ff77014d41beb98e5748f8f255a8/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:36.862595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734cbb3-063c-4a9e-a988-73ea1bf12c61 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.862595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a734cbb3-063c-4a9e-a988-73ea1bf12c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:36.875126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:36.895909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:36.907267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 805/1608] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:36.908079 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ea421-08b5-46aa-af74-4be968520597 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:36.922492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734cbb3-063c-4a9e-a988-73ea1bf12c61 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.923744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ea421-08b5-46aa-af74-4be968520597 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:36.926806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45189f-c84c-4d0c-b5e7-75c097e32ef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.927448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45189f-c84c-4d0c-b5e7-75c097e32ef7 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.927755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45189f-c84c-4d0c-b5e7-75c097e32ef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.928020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45189f-c84c-4d0c-b5e7-75c097e32ef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.984224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f291544-63dd-4be0-bcca-3ce88866e540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:36.985113 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f291544-63dd-4be0-bcca-3ce88866e540 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:36.985391 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f291544-63dd-4be0-bcca-3ce88866e540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:36.985673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f291544-63dd-4be0-bcca-3ce88866e540 admin admin] PATH_INFO: `/v3/projects/5cb09766b9784d50aad144d466a5d050/users/3b3f8de4a3b54d58806506b827cfa89a/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:36.992744 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45189f-c84c-4d0c-b5e7-75c097e32ef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:36.994697 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45189f-c84c-4d0c-b5e7-75c097e32ef7 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.000405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f291544-63dd-4be0-bcca-3ce88866e540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb09766b9784d50aad144d466a5d050, user_id=3b3f8de4a3b54d58806506b827cfa89a, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:37.004004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f291544-63dd-4be0-bcca-3ce88866e540 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.005249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 804/1609] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:37.006064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45189f-c84c-4d0c-b5e7-75c097e32ef7 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:37.015746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45189f-c84c-4d0c-b5e7-75c097e32ef7 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:37.020614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 806/1610] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:36 2022] PUT /identity/v3/projects/5cb09766b9784d50aad144d466a5d050/users/3b3f8de4a3b54d58806506b827cfa89a/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:37.021621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f291544-63dd-4be0-bcca-3ce88866e540 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:37.088449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dad3d-9d9a-4eff-bf2b-54987b89d46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:37.089382 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dad3d-9d9a-4eff-bf2b-54987b89d46b admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.089764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dad3d-9d9a-4eff-bf2b-54987b89d46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.090118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dad3d-9d9a-4eff-bf2b-54987b89d46b admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.090824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dad3d-9d9a-4eff-bf2b-54987b89d46b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:37.092783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dad3d-9d9a-4eff-bf2b-54987b89d46b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.099259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f291544-63dd-4be0-bcca-3ce88866e540 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:37.105219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200e65-fa78-4e50-b631-b7c8b6eded59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:37.105945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200e65-fa78-4e50-b631-b7c8b6eded59 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.106230 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200e65-fa78-4e50-b631-b7c8b6eded59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.106504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200e65-fa78-4e50-b631-b7c8b6eded59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.107254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c200e65-fa78-4e50-b631-b7c8b6eded59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:37.109162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c200e65-fa78-4e50-b631-b7c8b6eded59 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.130513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 805/1611] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:37 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:37.131690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dad3d-9d9a-4eff-bf2b-54987b89d46b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:37.143920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 807/1612] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:37 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:37.145291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200e65-fa78-4e50-b631-b7c8b6eded59 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:37.193424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:37.210776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200e65-fa78-4e50-b631-b7c8b6eded59 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:37.215300 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071f53c-3347-41b3-86cd-c1e81f9c29be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:37.215959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071f53c-3347-41b3-86cd-c1e81f9c29be service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.216242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071f53c-3347-41b3-86cd-c1e81f9c29be service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.216517 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071f53c-3347-41b3-86cd-c1e81f9c29be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.223547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dad3d-9d9a-4eff-bf2b-54987b89d46b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:37.243505 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:37.302418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306ab17-3c80-48fe-8a7c-c8ee763dea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:37.303404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306ab17-3c80-48fe-8a7c-c8ee763dea91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.303686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306ab17-3c80-48fe-8a7c-c8ee763dea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.304489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306ab17-3c80-48fe-8a7c-c8ee763dea91 admin admin] PATH_INFO: `/v3/projects/54ec292e2bcc4c24a926e7e289e4a113/users/ad5ebcb96a514abd962cf3047e539b02/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.332867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306ab17-3c80-48fe-8a7c-c8ee763dea91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ec292e2bcc4c24a926e7e289e4a113, user_id=ad5ebcb96a514abd962cf3047e539b02, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:37.337063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306ab17-3c80-48fe-8a7c-c8ee763dea91 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.338252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071f53c-3347-41b3-86cd-c1e81f9c29be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:37.340416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071f53c-3347-41b3-86cd-c1e81f9c29be service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.364022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 806/1613] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:37 2022] PUT /identity/v3/projects/54ec292e2bcc4c24a926e7e289e4a113/users/ad5ebcb96a514abd962cf3047e539b02/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:37.364022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da3f70-cc87-4100-9da7-293812b61f62 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.364022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da3f70-cc87-4100-9da7-293812b61f62 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.364022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da3f70-cc87-4100-9da7-293812b61f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.367799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 808/1614] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:37.375880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-92da3f70-cc87-4100-9da7-293812b61f62 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:37.377972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-92da3f70-cc87-4100-9da7-293812b61f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:37.383003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b375ee6-b923-4793-82e2-e376d4074d4c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.383298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b375ee6-b923-4793-82e2-e376d4074d4c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.383583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b375ee6-b923-4793-82e2-e376d4074d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.426340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0b375ee6-b923-4793-82e2-e376d4074d4c None None] MFA Rules not processed for user `02438387c012405eb258a65311c092a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:37.429232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0b375ee6-b923-4793-82e2-e376d4074d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:37.464392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 807/1615] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:37.467372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff5af2-a6b4-4b1d-b641-77e2f4d993fe None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.468051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff5af2-a6b4-4b1d-b641-77e2f4d993fe None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.468518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff5af2-a6b4-4b1d-b641-77e2f4d993fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.491564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-41ff5af2-a6b4-4b1d-b641-77e2f4d993fe None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:37.492965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-41ff5af2-a6b4-4b1d-b641-77e2f4d993fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:37.504842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 808/1616] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:37.504842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff5af2-a6b4-4b1d-b641-77e2f4d993fe None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:37.515150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff5af2-a6b4-4b1d-b641-77e2f4d993fe None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:37.524423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 809/1617] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:37.524423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b375ee6-b923-4793-82e2-e376d4074d4c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:37.527285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29fc3b-6bf5-4625-9dab-f5b4d2c0ec6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:37.528163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29fc3b-6bf5-4625-9dab-f5b4d2c0ec6b admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.528752 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29fc3b-6bf5-4625-9dab-f5b4d2c0ec6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.529203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29fc3b-6bf5-4625-9dab-f5b4d2c0ec6b admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.530255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29fc3b-6bf5-4625-9dab-f5b4d2c0ec6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:37.532362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29fc3b-6bf5-4625-9dab-f5b4d2c0ec6b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.562920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b375ee6-b923-4793-82e2-e376d4074d4c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:37.574928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 809/1618] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:37 2022] POST /identity/v3/users => generated 294 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:37.578920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfec8de-22a3-4633-84df-f3df355c10aa None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.580019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485650e-67f0-4404-a9b9-11bd6890281e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:37.580963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfec8de-22a3-4633-84df-f3df355c10aa None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.581233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfec8de-22a3-4633-84df-f3df355c10aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.583140 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485650e-67f0-4404-a9b9-11bd6890281e admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.583625 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485650e-67f0-4404-a9b9-11bd6890281e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.583910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485650e-67f0-4404-a9b9-11bd6890281e admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.584606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485650e-67f0-4404-a9b9-11bd6890281e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:37.587366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485650e-67f0-4404-a9b9-11bd6890281e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.626409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4dfec8de-22a3-4633-84df-f3df355c10aa None None] MFA Rules not processed for user `ad5ebcb96a514abd962cf3047e539b02`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:37.628802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4dfec8de-22a3-4633-84df-f3df355c10aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:37.638723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 810/1619] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:37 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:37.640563 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485650e-67f0-4404-a9b9-11bd6890281e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:37.707749 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:37.730110 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485650e-67f0-4404-a9b9-11bd6890281e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:37.735540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e35d8-7945-4c7b-b919-2c7963491201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:37.736324 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e35d8-7945-4c7b-b919-2c7963491201 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.737126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e35d8-7945-4c7b-b919-2c7963491201 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.738048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e35d8-7945-4c7b-b919-2c7963491201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.742966 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 810/1620] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:37 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:37.742966 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfec8de-22a3-4633-84df-f3df355c10aa None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:37.754781 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e35d8-7945-4c7b-b919-2c7963491201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:37.756474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e35d8-7945-4c7b-b919-2c7963491201 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.757613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfec8de-22a3-4633-84df-f3df355c10aa None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:37.763372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdf41b-5c06-48b8-bd40-fd401d0d26d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:37.764230 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdf41b-5c06-48b8-bd40-fd401d0d26d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.764651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdf41b-5c06-48b8-bd40-fd401d0d26d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.764937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdf41b-5c06-48b8-bd40-fd401d0d26d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.772274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 811/1621] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:37.774982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de47a9-591e-4c36-9901-85e68897d659 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.775383 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de47a9-591e-4c36-9901-85e68897d659 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.775718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de47a9-591e-4c36-9901-85e68897d659 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.777855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 812/1622] 10.222.0.46 () {54 vars in 961 bytes} [Thu May 26 18:17:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:17:37.778521 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdf41b-5c06-48b8-bd40-fd401d0d26d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:37.778835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de47a9-591e-4c36-9901-85e68897d659 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:37.780466 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdf41b-5c06-48b8-bd40-fd401d0d26d9 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.795406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de47a9-591e-4c36-9901-85e68897d659 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:37.798197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 811/1623] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:37.799127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdf41b-5c06-48b8-bd40-fd401d0d26d9 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:37.800980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2773a-e8be-40a2-8f75-13230a461aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:37.801774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2773a-e8be-40a2-8f75-13230a461aaf admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.802164 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2773a-e8be-40a2-8f75-13230a461aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.802537 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2773a-e8be-40a2-8f75-13230a461aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.803667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2773a-e8be-40a2-8f75-13230a461aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:37.806406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2773a-e8be-40a2-8f75-13230a461aaf admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.810997 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdf41b-5c06-48b8-bd40-fd401d0d26d9 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:37.819868 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c2db7-5d27-4a17-abb1-f386b1a04a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:37.820457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c2db7-5d27-4a17-abb1-f386b1a04a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.820949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c2db7-5d27-4a17-abb1-f386b1a04a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.821343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 813/1624] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:37.822388 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2773a-e8be-40a2-8f75-13230a461aaf admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:37.824612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c2db7-5d27-4a17-abb1-f386b1a04a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.825416 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c2db7-5d27-4a17-abb1-f386b1a04a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:37.828073 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c2db7-5d27-4a17-abb1-f386b1a04a04 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.845469 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 812/1625] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:37.846410 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d901e4d-4ba4-4fdd-bc40-1ad20268b52c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.846946 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d901e4d-4ba4-4fdd-bc40-1ad20268b52c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.847513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d901e4d-4ba4-4fdd-bc40-1ad20268b52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.855177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2773a-e8be-40a2-8f75-13230a461aaf admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:37.879713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51abf9-eb9a-42f2-bbed-180c671087bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:37.880371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51abf9-eb9a-42f2-bbed-180c671087bb admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:37.880653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51abf9-eb9a-42f2-bbed-180c671087bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:37.880926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51abf9-eb9a-42f2-bbed-180c671087bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:37.881680 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51abf9-eb9a-42f2-bbed-180c671087bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:37.882770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3d901e4d-4ba4-4fdd-bc40-1ad20268b52c None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:37.883843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51abf9-eb9a-42f2-bbed-180c671087bb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:37.884246 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3d901e4d-4ba4-4fdd-bc40-1ad20268b52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:37.954750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 814/1626] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:37 2022] POST /identity/v3/projects => generated 403 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:37.954750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51abf9-eb9a-42f2-bbed-180c671087bb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:37.981909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:37.988924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 813/1627] 10.222.0.46 () {58 vars in 1058 bytes} [Thu May 26 18:17:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:37.990198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d901e4d-4ba4-4fdd-bc40-1ad20268b52c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:38.005788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d901e4d-4ba4-4fdd-bc40-1ad20268b52c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:38.119609 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51abf9-eb9a-42f2-bbed-180c671087bb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:38.121667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6034c-a15b-4e42-8b17-14f76a5d283f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:38.123059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6034c-a15b-4e42-8b17-14f76a5d283f admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:38.123766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6034c-a15b-4e42-8b17-14f76a5d283f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:38.124187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6034c-a15b-4e42-8b17-14f76a5d283f admin admin] PATH_INFO: `/v3/projects/51eed2bfd5e84319bd601fb137512c17/users/0d7f5f27dd384fa09839f59d4b2b6576/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:38.131684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff57889-3eab-4a94-80f9-e2b78234891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:38.132979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff57889-3eab-4a94-80f9-e2b78234891f admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:38.132979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff57889-3eab-4a94-80f9-e2b78234891f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:38.135922 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff57889-3eab-4a94-80f9-e2b78234891f admin admin] PATH_INFO: `/v3/projects/b771aef331264e51bf044e5ce47ee693/users/8597d4ac1d904368a237032721543113/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:38.148013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6034c-a15b-4e42-8b17-14f76a5d283f admin admin] RBAC: Authorizing `identity:create_grant(project_id=51eed2bfd5e84319bd601fb137512c17, user_id=0d7f5f27dd384fa09839f59d4b2b6576, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:38.151044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6034c-a15b-4e42-8b17-14f76a5d283f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:38.177078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 814/1628] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:37 2022] PUT /identity/v3/projects/51eed2bfd5e84319bd601fb137512c17/users/0d7f5f27dd384fa09839f59d4b2b6576/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:38.178313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6034c-a15b-4e42-8b17-14f76a5d283f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:38.184166 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff57889-3eab-4a94-80f9-e2b78234891f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b771aef331264e51bf044e5ce47ee693, user_id=8597d4ac1d904368a237032721543113, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:38.188006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff57889-3eab-4a94-80f9-e2b78234891f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:38.217444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 815/1629] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:37 2022] PUT /identity/v3/projects/b771aef331264e51bf044e5ce47ee693/users/8597d4ac1d904368a237032721543113/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:38.220667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff57889-3eab-4a94-80f9-e2b78234891f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:38.288267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:38.337580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6034c-a15b-4e42-8b17-14f76a5d283f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:38.371111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff57889-3eab-4a94-80f9-e2b78234891f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:38.380084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264fe7a-c5ab-41a3-ad98-1127ccb11d1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:38.382882 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264fe7a-c5ab-41a3-ad98-1127ccb11d1f service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:38.383278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264fe7a-c5ab-41a3-ad98-1127ccb11d1f service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:38.383770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264fe7a-c5ab-41a3-ad98-1127ccb11d1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:38.478387 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1995f9c-32f7-44f5-b843-73569044fdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:38.479327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1995f9c-32f7-44f5-b843-73569044fdad admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:38.480152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1995f9c-32f7-44f5-b843-73569044fdad admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:38.480383 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1995f9c-32f7-44f5-b843-73569044fdad admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:38.481340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1995f9c-32f7-44f5-b843-73569044fdad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:38.486299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1995f9c-32f7-44f5-b843-73569044fdad admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:38.522711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 815/1630] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:38 2022] POST /identity/v3/users => generated 308 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:38.523739 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1995f9c-32f7-44f5-b843-73569044fdad admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:38.526275 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264fe7a-c5ab-41a3-ad98-1127ccb11d1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:38.527919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264fe7a-c5ab-41a3-ad98-1127ccb11d1f service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:38.537606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1995f9c-32f7-44f5-b843-73569044fdad admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:38.544994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc3c52-b6e2-4c1f-84b5-69a019d3e188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:38.546058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc3c52-b6e2-4c1f-84b5-69a019d3e188 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:38.546539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc3c52-b6e2-4c1f-84b5-69a019d3e188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:38.546922 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc3c52-b6e2-4c1f-84b5-69a019d3e188 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:38.547724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc3c52-b6e2-4c1f-84b5-69a019d3e188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:38.548809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 816/1631] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:38.549957 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264fe7a-c5ab-41a3-ad98-1127ccb11d1f service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:38.564447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc3c52-b6e2-4c1f-84b5-69a019d3e188 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:38.575618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264fe7a-c5ab-41a3-ad98-1127ccb11d1f service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:38.586853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 816/1632] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:38.587772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc3c52-b6e2-4c1f-84b5-69a019d3e188 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:38.588204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7afc71-976a-4dde-a3b4-04445a5ce80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:38.590233 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7afc71-976a-4dde-a3b4-04445a5ce80b admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:38.591717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7afc71-976a-4dde-a3b4-04445a5ce80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:38.593962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7afc71-976a-4dde-a3b4-04445a5ce80b admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:38.594964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7afc71-976a-4dde-a3b4-04445a5ce80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:38.599128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7afc71-976a-4dde-a3b4-04445a5ce80b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:38.602068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc3c52-b6e2-4c1f-84b5-69a019d3e188 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:38.607805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12dde2-2c72-46dc-ab07-c26403d4d8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:38.608643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12dde2-2c72-46dc-ab07-c26403d4d8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:38.609243 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12dde2-2c72-46dc-ab07-c26403d4d8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:38.609243 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12dde2-2c72-46dc-ab07-c26403d4d8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:38.610006 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12dde2-2c72-46dc-ab07-c26403d4d8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:38.612606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12dde2-2c72-46dc-ab07-c26403d4d8f3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:38.629796 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 817/1633] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:38.630681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12dde2-2c72-46dc-ab07-c26403d4d8f3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:38.635720 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 817/1634] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:38.638054 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7afc71-976a-4dde-a3b4-04445a5ce80b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:38.650798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12dde2-2c72-46dc-ab07-c26403d4d8f3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:38.657342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d62fc3-08c4-488d-b28a-d4088d639cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:38.660517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d62fc3-08c4-488d-b28a-d4088d639cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:38.660808 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d62fc3-08c4-488d-b28a-d4088d639cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:38.661164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d62fc3-08c4-488d-b28a-d4088d639cb7 admin admin] PATH_INFO: `/v3/projects/51eed2bfd5e84319bd601fb137512c17/users/0d7f5f27dd384fa09839f59d4b2b6576/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:38.663862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d62fc3-08c4-488d-b28a-d4088d639cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51eed2bfd5e84319bd601fb137512c17, user_id=0d7f5f27dd384fa09839f59d4b2b6576, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:38.664247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7afc71-976a-4dde-a3b4-04445a5ce80b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:38.667554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d62fc3-08c4-488d-b28a-d4088d639cb7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:38.678009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2079f-35de-4199-9bed-1d81aacedc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:38.679287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2079f-35de-4199-9bed-1d81aacedc88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:38.679745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2079f-35de-4199-9bed-1d81aacedc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:38.680179 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2079f-35de-4199-9bed-1d81aacedc88 admin admin] PATH_INFO: `/v3/projects/b771aef331264e51bf044e5ce47ee693/users/8597d4ac1d904368a237032721543113/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:38.683884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2079f-35de-4199-9bed-1d81aacedc88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b771aef331264e51bf044e5ce47ee693, user_id=8597d4ac1d904368a237032721543113, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:38.684806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 818/1635] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:38 2022] PUT /identity/v3/projects/51eed2bfd5e84319bd601fb137512c17/users/0d7f5f27dd384fa09839f59d4b2b6576/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:38.685810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d62fc3-08c4-488d-b28a-d4088d639cb7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:38.691326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2079f-35de-4199-9bed-1d81aacedc88 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:38.726411 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 818/1636] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:38 2022] PUT /identity/v3/projects/b771aef331264e51bf044e5ce47ee693/users/8597d4ac1d904368a237032721543113/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:38.727967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7772d-8193-4ccc-b74e-5f78c8c21e82 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:38.728255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7772d-8193-4ccc-b74e-5f78c8c21e82 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:38.728586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7772d-8193-4ccc-b74e-5f78c8c21e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:38.747957 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:38.789859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c2c7772d-8193-4ccc-b74e-5f78c8c21e82 None None] MFA Rules not processed for user `0d7f5f27dd384fa09839f59d4b2b6576`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:38.791084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c2c7772d-8193-4ccc-b74e-5f78c8c21e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:38.794218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d62fc3-08c4-488d-b28a-d4088d639cb7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:38.856865 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:38.890881 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8312e1-3ff0-4cc1-9241-61d36d3c7b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:38.894086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8312e1-3ff0-4cc1-9241-61d36d3c7b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:38.894904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8312e1-3ff0-4cc1-9241-61d36d3c7b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:38.895514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8312e1-3ff0-4cc1-9241-61d36d3c7b43 admin admin] PATH_INFO: `/v3/projects/fb23e39ef1d34d27a3a3f074c40d5ce6/users/d577d0dc0a424e3d88bb3ce5ce20b513/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:38.915823 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 819/1637] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:38 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:38.917635 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6771a3-7fbd-45e8-959d-816774d8ca3c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:38.918208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6771a3-7fbd-45e8-959d-816774d8ca3c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:38.918680 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6771a3-7fbd-45e8-959d-816774d8ca3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:38.932372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8312e1-3ff0-4cc1-9241-61d36d3c7b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb23e39ef1d34d27a3a3f074c40d5ce6, user_id=d577d0dc0a424e3d88bb3ce5ce20b513, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:38.935926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8312e1-3ff0-4cc1-9241-61d36d3c7b43 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:38.965971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 819/1638] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:38 2022] PUT /identity/v3/projects/fb23e39ef1d34d27a3a3f074c40d5ce6/users/d577d0dc0a424e3d88bb3ce5ce20b513/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:38.967989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8312e1-3ff0-4cc1-9241-61d36d3c7b43 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:38.977201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1c6771a3-7fbd-45e8-959d-816774d8ca3c None None] MFA Rules not processed for user `8597d4ac1d904368a237032721543113`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:38.991466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1c6771a3-7fbd-45e8-959d-816774d8ca3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:39.131863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8312e1-3ff0-4cc1-9241-61d36d3c7b43 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:39.140222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 820/1639] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:39.140222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6771a3-7fbd-45e8-959d-816774d8ca3c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:39.145598 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba271de-76ae-496b-849f-8e4ee0c03428 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:39.147288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba271de-76ae-496b-849f-8e4ee0c03428 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:39.147805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba271de-76ae-496b-849f-8e4ee0c03428 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:39.148300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba271de-76ae-496b-849f-8e4ee0c03428 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:39.258724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba271de-76ae-496b-849f-8e4ee0c03428 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:39.261235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba271de-76ae-496b-849f-8e4ee0c03428 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:39.269154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6771a3-7fbd-45e8-959d-816774d8ca3c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:39.276117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 820/1640] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:39.278516 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808064a9-3e88-4bc8-81b5-d3433d83a089 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:39.278914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808064a9-3e88-4bc8-81b5-d3433d83a089 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:39.279297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808064a9-3e88-4bc8-81b5-d3433d83a089 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:39.281327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 821/1641] 10.222.0.46 () {54 vars in 912 bytes} [Thu May 26 18:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:17:39.282140 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702ef00-099c-4d36-ac5b-69faf8e09959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:39.282299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808064a9-3e88-4bc8-81b5-d3433d83a089 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:39.284089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702ef00-099c-4d36-ac5b-69faf8e09959 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:39.284829 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702ef00-099c-4d36-ac5b-69faf8e09959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:39.285276 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702ef00-099c-4d36-ac5b-69faf8e09959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:39.286532 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702ef00-099c-4d36-ac5b-69faf8e09959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:39.288682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702ef00-099c-4d36-ac5b-69faf8e09959 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:39.296828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808064a9-3e88-4bc8-81b5-d3433d83a089 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:39.302642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98739aa8-db7d-4f03-960d-5a4b331f7dbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:39.303437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98739aa8-db7d-4f03-960d-5a4b331f7dbb service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:39.303829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98739aa8-db7d-4f03-960d-5a4b331f7dbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:39.304194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98739aa8-db7d-4f03-960d-5a4b331f7dbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:39.307384 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 821/1642] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:39.309162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34e371-e361-422d-8791-8657b5ef5648 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:39.309634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34e371-e361-422d-8791-8657b5ef5648 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:39.310374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34e371-e361-422d-8791-8657b5ef5648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:39.353401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bf34e371-e361-422d-8791-8657b5ef5648 None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:39.354644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bf34e371-e361-422d-8791-8657b5ef5648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:39.365336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:39.439155 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98739aa8-db7d-4f03-960d-5a4b331f7dbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:39.441917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98739aa8-db7d-4f03-960d-5a4b331f7dbb service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:39.462849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 822/1643] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:17:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:39.463874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34e371-e361-422d-8791-8657b5ef5648 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:39.470318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 822/1644] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:39.471701 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98739aa8-db7d-4f03-960d-5a4b331f7dbb service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:39.475549 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34e371-e361-422d-8791-8657b5ef5648 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:39.476212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-98739aa8-db7d-4f03-960d-5a4b331f7dbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:39.480804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b0104-6ab5-4104-8cd0-69b47b61d536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:39.481832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b0104-6ab5-4104-8cd0-69b47b61d536 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:39.482012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b0104-6ab5-4104-8cd0-69b47b61d536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:39.482443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b0104-6ab5-4104-8cd0-69b47b61d536 admin admin] PATH_INFO: `/v3/projects/fb23e39ef1d34d27a3a3f074c40d5ce6/users/d577d0dc0a424e3d88bb3ce5ce20b513/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:39.484449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b0104-6ab5-4104-8cd0-69b47b61d536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb23e39ef1d34d27a3a3f074c40d5ce6, user_id=d577d0dc0a424e3d88bb3ce5ce20b513, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:39.487474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b0104-6ab5-4104-8cd0-69b47b61d536 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:39.501639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 823/1645] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:39 2022] PUT /identity/v3/projects/fb23e39ef1d34d27a3a3f074c40d5ce6/users/d577d0dc0a424e3d88bb3ce5ce20b513/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:39.514206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa75fde-3a48-42dd-8e08-77c026d69c80 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:39.514456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa75fde-3a48-42dd-8e08-77c026d69c80 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:39.514736 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa75fde-3a48-42dd-8e08-77c026d69c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:39.532314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:39.550990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-baa75fde-3a48-42dd-8e08-77c026d69c80 None None] MFA Rules not processed for user `d577d0dc0a424e3d88bb3ce5ce20b513`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:39.552929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-baa75fde-3a48-42dd-8e08-77c026d69c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:39.635711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98739aa8-db7d-4f03-960d-5a4b331f7dbb service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:39.702553 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 824/1646] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:39 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:39.756167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f965a-680f-4a15-8860-e03d1ce022d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a67120485f74f8bb0ff188cd3abd21c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:39.757085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f965a-680f-4a15-8860-e03d1ce022d7 service glance] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:39.757478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f965a-680f-4a15-8860-e03d1ce022d7 service glance] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:39.758066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f965a-680f-4a15-8860-e03d1ce022d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:39.812850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa75fde-3a48-42dd-8e08-77c026d69c80 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:39.951306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f965a-680f-4a15-8860-e03d1ce022d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:39.955996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f965a-680f-4a15-8860-e03d1ce022d7 service glance] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:39.966006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa75fde-3a48-42dd-8e08-77c026d69c80 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:39.974910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6fe8e-298a-47e1-b533-c968fd4ef722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:39.975738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6fe8e-298a-47e1-b533-c968fd4ef722 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:39.976948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6fe8e-298a-47e1-b533-c968fd4ef722 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:39.977453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6fe8e-298a-47e1-b533-c968fd4ef722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:39.985700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 823/1647] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:39.998695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6fe8e-298a-47e1-b533-c968fd4ef722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:40.000530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6fe8e-298a-47e1-b533-c968fd4ef722 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:40.021472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 825/1648] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:39 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:40.512315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f965a-680f-4a15-8860-e03d1ce022d7 service glance] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:40.749777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f965a-680f-4a15-8860-e03d1ce022d7 service glance] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:40.768930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c01f0-1b0c-4108-ac71-9aef683b29f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:40.770089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c01f0-1b0c-4108-ac71-9aef683b29f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:40.770479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c01f0-1b0c-4108-ac71-9aef683b29f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:40.770855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c01f0-1b0c-4108-ac71-9aef683b29f6 admin admin] PATH_INFO: `/v3/users/ad5ebcb96a514abd962cf3047e539b02` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:40.772242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c01f0-1b0c-4108-ac71-9aef683b29f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad5ebcb96a514abd962cf3047e539b02)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:40.773493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-407c01f0-1b0c-4108-ac71-9aef683b29f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:17:40.773493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: warnings.warn(msg) May 26 18:17:40.773493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: May 26 18:17:40.775030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c01f0-1b0c-4108-ac71-9aef683b29f6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:40.811453 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:40.937965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-407c01f0-1b0c-4108-ac71-9aef683b29f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad5ebcb96a514abd962cf3047e539b02'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:40.965708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-407c01f0-1b0c-4108-ac71-9aef683b29f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad5ebcb96a514abd962cf3047e539b02'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:40.966473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-407c01f0-1b0c-4108-ac71-9aef683b29f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad5ebcb96a514abd962cf3047e539b02'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:40.967209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-407c01f0-1b0c-4108-ac71-9aef683b29f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad5ebcb96a514abd962cf3047e539b02'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:40.980584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6fe8e-298a-47e1-b533-c968fd4ef722 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:40.994323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6fe8e-298a-47e1-b533-c968fd4ef722 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:41.003930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83fc5c-ef77-4e0b-8fbf-ebc9a42012f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:41.006495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83fc5c-ef77-4e0b-8fbf-ebc9a42012f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:41.007000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83fc5c-ef77-4e0b-8fbf-ebc9a42012f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:41.007421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83fc5c-ef77-4e0b-8fbf-ebc9a42012f2 admin admin] PATH_INFO: `/v3/users/b9de367933b64fb0833d7ae095e02c7f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:41.009469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83fc5c-ef77-4e0b-8fbf-ebc9a42012f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9de367933b64fb0833d7ae095e02c7f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:41.011824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83fc5c-ef77-4e0b-8fbf-ebc9a42012f2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:41.038230 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:41.062668 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-407c01f0-1b0c-4108-ac71-9aef683b29f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad5ebcb96a514abd962cf3047e539b02'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:41.139460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 824/1649] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:17:40 2022] DELETE /identity/v3/users/ad5ebcb96a514abd962cf3047e539b02 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:41.143917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f12bc2-e725-4974-9102-b5d7b3c283d1 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:41.144117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f12bc2-e725-4974-9102-b5d7b3c283d1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:41.144450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f12bc2-e725-4974-9102-b5d7b3c283d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:41.161027 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6c83fc5c-ef77-4e0b-8fbf-ebc9a42012f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9de367933b64fb0833d7ae095e02c7f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:41.193175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6c83fc5c-ef77-4e0b-8fbf-ebc9a42012f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9de367933b64fb0833d7ae095e02c7f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:41.202461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f2f12bc2-e725-4974-9102-b5d7b3c283d1 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:41.208025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f2f12bc2-e725-4974-9102-b5d7b3c283d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:41.213546 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6c83fc5c-ef77-4e0b-8fbf-ebc9a42012f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9de367933b64fb0833d7ae095e02c7f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:41.214395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6c83fc5c-ef77-4e0b-8fbf-ebc9a42012f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9de367933b64fb0833d7ae095e02c7f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:41.224962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6c83fc5c-ef77-4e0b-8fbf-ebc9a42012f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9de367933b64fb0833d7ae095e02c7f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:41.235346 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 826/1650] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:17:40 2022] DELETE /identity/v3/users/b9de367933b64fb0833d7ae095e02c7f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:41.237224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f598e-37ea-4046-a4f4-51a4e71cefe0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:41.238170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f598e-37ea-4046-a4f4-51a4e71cefe0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:41.238523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f598e-37ea-4046-a4f4-51a4e71cefe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:41.304767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-513f598e-37ea-4046-a4f4-51a4e71cefe0 None None] MFA Rules not processed for user `96a0ff77014d41beb98e5748f8f255a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:41.306867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-513f598e-37ea-4046-a4f4-51a4e71cefe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:41.384915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 825/1651] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:41.386077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d13d12-90de-4db7-8089-0791e60b920d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:41.386367 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d13d12-90de-4db7-8089-0791e60b920d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:41.386641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d13d12-90de-4db7-8089-0791e60b920d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:41.419088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 827/1652] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:41 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:41.419687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568e8ba-cf9f-48ef-945d-338df1a4429e None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:41.420363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568e8ba-cf9f-48ef-945d-338df1a4429e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:41.420547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568e8ba-cf9f-48ef-945d-338df1a4429e None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:41.422391 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 828/1653] 10.222.0.46 () {54 vars in 912 bytes} [Thu May 26 18:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:17:41.423715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568e8ba-cf9f-48ef-945d-338df1a4429e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:41.425878 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c568e8ba-cf9f-48ef-945d-338df1a4429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:41.446097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-60d13d12-90de-4db7-8089-0791e60b920d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:41.449343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-60d13d12-90de-4db7-8089-0791e60b920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:41.602586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568e8ba-cf9f-48ef-945d-338df1a4429e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:41.615389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83667a6b-1f4a-45bb-89d2-084bdd5990c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:41.616061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83667a6b-1f4a-45bb-89d2-084bdd5990c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:41.616344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83667a6b-1f4a-45bb-89d2-084bdd5990c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:41.616621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83667a6b-1f4a-45bb-89d2-084bdd5990c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:41.670058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 826/1654] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:41.678160 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d13d12-90de-4db7-8089-0791e60b920d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:41.682275 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-60d13d12-90de-4db7-8089-0791e60b920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:41.724082 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:41.756029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d13d12-90de-4db7-8089-0791e60b920d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:41.761682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f14c5-7a2b-4922-91a7-0c8d6b684fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:41.762619 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f14c5-7a2b-4922-91a7-0c8d6b684fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:41.763908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f14c5-7a2b-4922-91a7-0c8d6b684fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:41.763908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f14c5-7a2b-4922-91a7-0c8d6b684fa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:41.765035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f14c5-7a2b-4922-91a7-0c8d6b684fa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:41.766798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f14c5-7a2b-4922-91a7-0c8d6b684fa5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:41.812732 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83667a6b-1f4a-45bb-89d2-084bdd5990c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:41.816048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83667a6b-1f4a-45bb-89d2-084bdd5990c4 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:41.825253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 827/1655] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:41 2022] POST /identity/v3/projects => generated 405 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:41.827062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958dafa-748c-41cb-9dbe-a2b8c1e81393 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:41.827529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958dafa-748c-41cb-9dbe-a2b8c1e81393 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:41.828221 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958dafa-748c-41cb-9dbe-a2b8c1e81393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:41.839663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 829/1656] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:41.867336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83667a6b-1f4a-45bb-89d2-084bdd5990c4 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:41.867336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83667a6b-1f4a-45bb-89d2-084bdd5990c4 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:41.888073 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:41.955137 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a958dafa-748c-41cb-9dbe-a2b8c1e81393 None None] MFA Rules not processed for user `f668c4c6745e4f1eae20d8a3a12cf0cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:41.957328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a958dafa-748c-41cb-9dbe-a2b8c1e81393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:42.025766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae7d42-ce3a-4637-ab74-400af970c447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:42.027671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae7d42-ce3a-4637-ab74-400af970c447 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:42.030064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae7d42-ce3a-4637-ab74-400af970c447 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:42.030385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae7d42-ce3a-4637-ab74-400af970c447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:42.122764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 828/1657] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:17:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:42.124280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958dafa-748c-41cb-9dbe-a2b8c1e81393 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:42.224542 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae7d42-ce3a-4637-ab74-400af970c447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:42.233366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae7d42-ce3a-4637-ab74-400af970c447 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:42.268910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 830/1658] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:42.311420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958dafa-748c-41cb-9dbe-a2b8c1e81393 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:42.321348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d46a26-1617-494c-8377-d3d573b69934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:42.322241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d46a26-1617-494c-8377-d3d573b69934 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:42.325472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d46a26-1617-494c-8377-d3d573b69934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:42.325923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d46a26-1617-494c-8377-d3d573b69934 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:42.326884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d46a26-1617-494c-8377-d3d573b69934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:42.329362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d46a26-1617-494c-8377-d3d573b69934 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:42.357533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 829/1659] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:42 2022] POST /identity/v3/users => generated 309 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:42.361418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae7d42-ce3a-4637-ab74-400af970c447 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:42.385072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae7d42-ce3a-4637-ab74-400af970c447 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:42.395429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306d1e7-3c19-4380-b9b9-2bd91c56f5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:42.396117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306d1e7-3c19-4380-b9b9-2bd91c56f5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:42.396866 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306d1e7-3c19-4380-b9b9-2bd91c56f5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:42.396866 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306d1e7-3c19-4380-b9b9-2bd91c56f5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:42.397604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306d1e7-3c19-4380-b9b9-2bd91c56f5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:42.399262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306d1e7-3c19-4380-b9b9-2bd91c56f5f7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:42.406629 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d46a26-1617-494c-8377-d3d573b69934 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:42.407911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-83d46a26-1617-494c-8377-d3d573b69934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:42.436497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 831/1660] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:42.438112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d46a26-1617-494c-8377-d3d573b69934 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:42.438462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306d1e7-3c19-4380-b9b9-2bd91c56f5f7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:42.448611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735897c-7b6b-40f5-9be6-073a0cd06a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:42.450905 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735897c-7b6b-40f5-9be6-073a0cd06a1f service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:42.451975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735897c-7b6b-40f5-9be6-073a0cd06a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:42.455231 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735897c-7b6b-40f5-9be6-073a0cd06a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:42.479127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735897c-7b6b-40f5-9be6-073a0cd06a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:42.482511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735897c-7b6b-40f5-9be6-073a0cd06a1f service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:42.495855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306d1e7-3c19-4380-b9b9-2bd91c56f5f7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:42.503551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 830/1661] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:42.536979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7271cdc-7ea8-4fd9-a6ca-5f1fdb1bf975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:42.546563 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7271cdc-7ea8-4fd9-a6ca-5f1fdb1bf975 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:42.546867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7271cdc-7ea8-4fd9-a6ca-5f1fdb1bf975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:42.547148 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7271cdc-7ea8-4fd9-a6ca-5f1fdb1bf975 admin admin] PATH_INFO: `/v3/projects/9283da8d71e843b6a92d56858470ddb2/users/896add8d2ac54725a350b86ac87897a5/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:42.600049 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7271cdc-7ea8-4fd9-a6ca-5f1fdb1bf975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9283da8d71e843b6a92d56858470ddb2, user_id=896add8d2ac54725a350b86ac87897a5, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:42.603073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7271cdc-7ea8-4fd9-a6ca-5f1fdb1bf975 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:42.646991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 832/1662] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:42 2022] PUT /identity/v3/projects/9283da8d71e843b6a92d56858470ddb2/users/896add8d2ac54725a350b86ac87897a5/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:42.650875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735897c-7b6b-40f5-9be6-073a0cd06a1f service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:42.682325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735897c-7b6b-40f5-9be6-073a0cd06a1f service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:42.789132 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4170d4-9f7c-466d-b21c-62490e916fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:42.790091 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4170d4-9f7c-466d-b21c-62490e916fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:42.790483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4170d4-9f7c-466d-b21c-62490e916fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:42.790887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4170d4-9f7c-466d-b21c-62490e916fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:42.791839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4170d4-9f7c-466d-b21c-62490e916fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:42.793709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4170d4-9f7c-466d-b21c-62490e916fa4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:42.808024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 831/1663] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:42.812133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7271cdc-7ea8-4fd9-a6ca-5f1fdb1bf975 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:42.825270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7271cdc-7ea8-4fd9-a6ca-5f1fdb1bf975 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:42.836414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fb617-3c1a-41d7-994f-8a093c96a654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:42.837695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fb617-3c1a-41d7-994f-8a093c96a654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:42.838344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fb617-3c1a-41d7-994f-8a093c96a654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:42.838713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fb617-3c1a-41d7-994f-8a093c96a654 admin admin] PATH_INFO: `/v3/projects/9283da8d71e843b6a92d56858470ddb2/users/896add8d2ac54725a350b86ac87897a5/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:42.841263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fb617-3c1a-41d7-994f-8a093c96a654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9283da8d71e843b6a92d56858470ddb2, user_id=896add8d2ac54725a350b86ac87897a5, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:42.847774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fb617-3c1a-41d7-994f-8a093c96a654 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:42.869148 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 833/1664] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:42 2022] PUT /identity/v3/projects/9283da8d71e843b6a92d56858470ddb2/users/896add8d2ac54725a350b86ac87897a5/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:43.663197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4170d4-9f7c-466d-b21c-62490e916fa4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:43.667663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5e4170d4-9f7c-466d-b21c-62490e916fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:43.704818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4170d4-9f7c-466d-b21c-62490e916fa4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:43.760770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:43.821043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fb617-3c1a-41d7-994f-8a093c96a654 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:43.823406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f82fb617-3c1a-41d7-994f-8a093c96a654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:43.902893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:43.903778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:43.904057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:43.908193 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:43.913855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] PATH_INFO: `/v3/projects/b6d18f4b036f492da77cb37f19c75827` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:43.919461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d18f4b036f492da77cb37f19c75827)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:43.921708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:43.969598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fb617-3c1a-41d7-994f-8a093c96a654 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:43.978460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:43.981334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:43.981334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:43.981780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] PATH_INFO: `/v3/projects/54ec292e2bcc4c24a926e7e289e4a113` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:43.984791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] RBAC: Authorizing `identity:delete_project(project_id=54ec292e2bcc4c24a926e7e289e4a113)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:43.988083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:17:43.988083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: warnings.warn(msg) May 26 18:17:43.988083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: May 26 18:17:43.989847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:44.020863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d18f4b036f492da77cb37f19c75827', 'request_id': 'req-bd242812-0b09-4aa9-bc23-2572f1f5d801'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.036843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d18f4b036f492da77cb37f19c75827', 'request_id': 'req-bd242812-0b09-4aa9-bc23-2572f1f5d801'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.044065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d18f4b036f492da77cb37f19c75827', 'request_id': 'req-bd242812-0b09-4aa9-bc23-2572f1f5d801'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.071776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] The token cache is being invalidate because project b6d18f4b036f492da77cb37f19c75827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:17:44.072072 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.073049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.074953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 832/1665] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:17:43 2022] DELETE /identity/v3/projects/b6d18f4b036f492da77cb37f19c75827 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:44.079584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:44.081847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:44.089234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54ec292e2bcc4c24a926e7e289e4a113', 'request_id': 'req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.113157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54ec292e2bcc4c24a926e7e289e4a113', 'request_id': 'req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.148951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54ec292e2bcc4c24a926e7e289e4a113', 'request_id': 'req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.175927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] The token cache is being invalidate because project 54ec292e2bcc4c24a926e7e289e4a113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:17:44.176225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.180549 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.187528 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 834/1666] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:17:43 2022] DELETE /identity/v3/projects/54ec292e2bcc4c24a926e7e289e4a113 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:44.189139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:44.192986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:44.248673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd242812-0b09-4aa9-bc23-2572f1f5d801 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:44.367534 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1089e0-4a5d-4a9a-a55d-c01e35db283a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:44.375327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3246d-580f-48a8-95d7-7afb90b65b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:44.376416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3246d-580f-48a8-95d7-7afb90b65b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:44.376798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3246d-580f-48a8-95d7-7afb90b65b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:44.377214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3246d-580f-48a8-95d7-7afb90b65b70 admin admin] PATH_INFO: `/v3/users/d577d0dc0a424e3d88bb3ce5ce20b513` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:44.381083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3246d-580f-48a8-95d7-7afb90b65b70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d577d0dc0a424e3d88bb3ce5ce20b513)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:44.393458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3246d-580f-48a8-95d7-7afb90b65b70 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:44.578329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bec3246d-580f-48a8-95d7-7afb90b65b70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd577d0dc0a424e3d88bb3ce5ce20b513'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.608105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bec3246d-580f-48a8-95d7-7afb90b65b70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd577d0dc0a424e3d88bb3ce5ce20b513'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.628770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:44.660165 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bec3246d-580f-48a8-95d7-7afb90b65b70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd577d0dc0a424e3d88bb3ce5ce20b513'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.660947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bec3246d-580f-48a8-95d7-7afb90b65b70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd577d0dc0a424e3d88bb3ce5ce20b513'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.705221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bec3246d-580f-48a8-95d7-7afb90b65b70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd577d0dc0a424e3d88bb3ce5ce20b513'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:44.733242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 835/1667] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:17:44 2022] DELETE /identity/v3/users/d577d0dc0a424e3d88bb3ce5ce20b513 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:44.743995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3c4e5-ccf8-423e-96f6-62ee4ced6181 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:44.746641 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3c4e5-ccf8-423e-96f6-62ee4ced6181 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:44.752214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3c4e5-ccf8-423e-96f6-62ee4ced6181 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:44.760347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 836/1668] 10.222.0.46 () {54 vars in 960 bytes} [Thu May 26 18:17:44 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:17:44.768487 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb81ac-1fbf-46eb-be0e-97b9bf2d9de6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:44.769624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb81ac-1fbf-46eb-be0e-97b9bf2d9de6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:44.770717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb81ac-1fbf-46eb-be0e-97b9bf2d9de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:44.858782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:44.867317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b8bb81ac-1fbf-46eb-be0e-97b9bf2d9de6 None None] MFA Rules not processed for user `7ee40bcb7905490c9ad3e39e7a93b122`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:44.877339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b8bb81ac-1fbf-46eb-be0e-97b9bf2d9de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:44.881980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22465c0-8975-4955-9e8c-5ddb92bfc57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:44.883690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22465c0-8975-4955-9e8c-5ddb92bfc57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:44.883978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22465c0-8975-4955-9e8c-5ddb92bfc57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:44.884252 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22465c0-8975-4955-9e8c-5ddb92bfc57a admin admin] PATH_INFO: `/v3/users/0d7f5f27dd384fa09839f59d4b2b6576` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:44.885513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22465c0-8975-4955-9e8c-5ddb92bfc57a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7f5f27dd384fa09839f59d4b2b6576)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:44.893600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22465c0-8975-4955-9e8c-5ddb92bfc57a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:45.064445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f22465c0-8975-4955-9e8c-5ddb92bfc57a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7f5f27dd384fa09839f59d4b2b6576'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:45.067665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 837/1669] 10.222.0.46 () {58 vars in 1057 bytes} [Thu May 26 18:17:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:17:45.094481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f22465c0-8975-4955-9e8c-5ddb92bfc57a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7f5f27dd384fa09839f59d4b2b6576'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:45.097032 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f22465c0-8975-4955-9e8c-5ddb92bfc57a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7f5f27dd384fa09839f59d4b2b6576'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:45.098205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb81ac-1fbf-46eb-be0e-97b9bf2d9de6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:45.099783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f22465c0-8975-4955-9e8c-5ddb92bfc57a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7f5f27dd384fa09839f59d4b2b6576'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:45.115446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb81ac-1fbf-46eb-be0e-97b9bf2d9de6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:45.134806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54197185-1401-44c3-b593-8fbe1b0c579a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:45.135471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54197185-1401-44c3-b593-8fbe1b0c579a service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:45.135748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54197185-1401-44c3-b593-8fbe1b0c579a service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:45.136017 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54197185-1401-44c3-b593-8fbe1b0c579a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:45.137767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f22465c0-8975-4955-9e8c-5ddb92bfc57a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7f5f27dd384fa09839f59d4b2b6576'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:45.144999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-54197185-1401-44c3-b593-8fbe1b0c579a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:45.178869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54197185-1401-44c3-b593-8fbe1b0c579a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:45.180493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54197185-1401-44c3-b593-8fbe1b0c579a service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:45.191165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 833/1670] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:17:44 2022] DELETE /identity/v3/users/0d7f5f27dd384fa09839f59d4b2b6576 => generated 0 bytes in 1109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:45.222953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 838/1671] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:45.243704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b177875-b3e6-4cac-9821-7e95edf83ba2 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:45.244002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b177875-b3e6-4cac-9821-7e95edf83ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:45.244280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b177875-b3e6-4cac-9821-7e95edf83ba2 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:45.246827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 834/1672] 10.222.0.46 () {54 vars in 961 bytes} [Thu May 26 18:17:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:17:45.257214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84743d13-3989-4960-bc7d-384a169afe71 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:45.258234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84743d13-3989-4960-bc7d-384a169afe71 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:45.258497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84743d13-3989-4960-bc7d-384a169afe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:45.294989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-84743d13-3989-4960-bc7d-384a169afe71 None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:45.296506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-84743d13-3989-4960-bc7d-384a169afe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:45.451479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fa446-9743-45c5-a020-663b424d8484 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:45.451836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fa446-9743-45c5-a020-663b424d8484 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:45.452115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fa446-9743-45c5-a020-663b424d8484 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:45.452224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 839/1673] 10.222.0.46 () {58 vars in 1058 bytes} [Thu May 26 18:17:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:45.454077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 835/1674] 10.222.0.46 () {54 vars in 912 bytes} [Thu May 26 18:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:17:45.462867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed6d5d-d442-4b9b-bb37-02e8a180199a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:45.463051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed6d5d-d442-4b9b-bb37-02e8a180199a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:45.463321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed6d5d-d442-4b9b-bb37-02e8a180199a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:45.498557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4aed6d5d-d442-4b9b-bb37-02e8a180199a None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:45.499807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4aed6d5d-d442-4b9b-bb37-02e8a180199a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:45.530984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 840/1675] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:17:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:45.639674 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fa446-9743-45c5-a020-663b424d8484 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:45.643290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-140fa446-9743-45c5-a020-663b424d8484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:45.813395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fa446-9743-45c5-a020-663b424d8484 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:45.823105 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5afbe-4947-4aa5-8aa3-d5051fdd71bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:45.824646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5afbe-4947-4aa5-8aa3-d5051fdd71bf service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:45.824929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5afbe-4947-4aa5-8aa3-d5051fdd71bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:45.827647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5afbe-4947-4aa5-8aa3-d5051fdd71bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:45.872355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5afbe-4947-4aa5-8aa3-d5051fdd71bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:45.874020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5afbe-4947-4aa5-8aa3-d5051fdd71bf service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:45.894669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 836/1676] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:45.903213 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed6d5d-d442-4b9b-bb37-02e8a180199a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:45.910659 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4aed6d5d-d442-4b9b-bb37-02e8a180199a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:46.066575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:46.076429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed6d5d-d442-4b9b-bb37-02e8a180199a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:46.090254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9c251-a0ea-4a02-b22b-1c9106cec55a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:46.090918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9c251-a0ea-4a02-b22b-1c9106cec55a service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:46.093218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9c251-a0ea-4a02-b22b-1c9106cec55a service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:46.093707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9c251-a0ea-4a02-b22b-1c9106cec55a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:46.098193 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bbd9c251-a0ea-4a02-b22b-1c9106cec55a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:46.299445 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9c251-a0ea-4a02-b22b-1c9106cec55a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:46.304478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9c251-a0ea-4a02-b22b-1c9106cec55a service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:46.340063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 841/1677] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:46.924067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5afbe-4947-4aa5-8aa3-d5051fdd71bf service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:46.925918 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-98d5afbe-4947-4aa5-8aa3-d5051fdd71bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:47.105493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5afbe-4947-4aa5-8aa3-d5051fdd71bf service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:47.116809 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3827a-6fd5-4125-a6fe-c77fc687b384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:47.117759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3827a-6fd5-4125-a6fe-c77fc687b384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:47.118049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3827a-6fd5-4125-a6fe-c77fc687b384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:47.118319 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3827a-6fd5-4125-a6fe-c77fc687b384 admin admin] PATH_INFO: `/v3/projects/51eed2bfd5e84319bd601fb137512c17` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:47.119854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3827a-6fd5-4125-a6fe-c77fc687b384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51eed2bfd5e84319bd601fb137512c17)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:47.125903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3827a-6fd5-4125-a6fe-c77fc687b384 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:47.165270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312d7b2-d304-410b-a769-ed8c678d6c9c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:47.165270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312d7b2-d304-410b-a769-ed8c678d6c9c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:47.166077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312d7b2-d304-410b-a769-ed8c678d6c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:47.210125 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-6312d7b2-d304-410b-a769-ed8c678d6c9c None None] MFA Rules not processed for user `896add8d2ac54725a350b86ac87897a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:47.211992 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6312d7b2-d304-410b-a769-ed8c678d6c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:47.224123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:47.332282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dda3827a-6fd5-4125-a6fe-c77fc687b384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51eed2bfd5e84319bd601fb137512c17', 'request_id': 'req-dda3827a-6fd5-4125-a6fe-c77fc687b384'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:47.356925 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dda3827a-6fd5-4125-a6fe-c77fc687b384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51eed2bfd5e84319bd601fb137512c17', 'request_id': 'req-dda3827a-6fd5-4125-a6fe-c77fc687b384'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:47.387681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dda3827a-6fd5-4125-a6fe-c77fc687b384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51eed2bfd5e84319bd601fb137512c17', 'request_id': 'req-dda3827a-6fd5-4125-a6fe-c77fc687b384'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:47.431912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dda3827a-6fd5-4125-a6fe-c77fc687b384 admin admin] The token cache is being invalidate because project 51eed2bfd5e84319bd601fb137512c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:17:47.431912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dda3827a-6fd5-4125-a6fe-c77fc687b384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:47.438146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dda3827a-6fd5-4125-a6fe-c77fc687b384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:47.447965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 837/1678] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:17:46 2022] DELETE /identity/v3/projects/51eed2bfd5e84319bd601fb137512c17 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:47.448084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 842/1679] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:47 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:47.451149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b26e92-95ea-46aa-8e08-814ef36694b5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:47.452337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b26e92-95ea-46aa-8e08-814ef36694b5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:47.452709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b26e92-95ea-46aa-8e08-814ef36694b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:47.480917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f1b26e92-95ea-46aa-8e08-814ef36694b5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:47.482711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f1b26e92-95ea-46aa-8e08-814ef36694b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:47.676071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 838/1680] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:17:47.678447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79288a34-bc6d-4336-8547-8c99be55e171 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:47.678900 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79288a34-bc6d-4336-8547-8c99be55e171 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:47.679247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79288a34-bc6d-4336-8547-8c99be55e171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:47.724603 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-79288a34-bc6d-4336-8547-8c99be55e171 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:47.730749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-79288a34-bc6d-4336-8547-8c99be55e171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:47.769799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 843/1681] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:47.772370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b26e92-95ea-46aa-8e08-814ef36694b5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:47.783777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b26e92-95ea-46aa-8e08-814ef36694b5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:47.789539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15266629-f6da-4ca3-a57e-530db589cb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:47.790736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15266629-f6da-4ca3-a57e-530db589cb01 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:47.791153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15266629-f6da-4ca3-a57e-530db589cb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:47.791591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15266629-f6da-4ca3-a57e-530db589cb01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:47.792456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15266629-f6da-4ca3-a57e-530db589cb01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:17:47.792921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15266629-f6da-4ca3-a57e-530db589cb01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:47.795809 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15266629-f6da-4ca3-a57e-530db589cb01 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:47.813022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 839/1682] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:17:47.818197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79288a34-bc6d-4336-8547-8c99be55e171 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:47.828376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79288a34-bc6d-4336-8547-8c99be55e171 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:47.840928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5cadf-181a-4ab7-82b1-eec75b1304be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:47.841868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5cadf-181a-4ab7-82b1-eec75b1304be admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:47.842217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5cadf-181a-4ab7-82b1-eec75b1304be admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:47.842572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5cadf-181a-4ab7-82b1-eec75b1304be admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:47.843326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5cadf-181a-4ab7-82b1-eec75b1304be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:47.845390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5cadf-181a-4ab7-82b1-eec75b1304be admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:47.916573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 844/1683] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:47 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:47.921106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15266629-f6da-4ca3-a57e-530db589cb01 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:47.938211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15266629-f6da-4ca3-a57e-530db589cb01 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:48.075823 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5cadf-181a-4ab7-82b1-eec75b1304be admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:48.078561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3dd5cadf-181a-4ab7-82b1-eec75b1304be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:48.132197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1700e5-58b7-4842-a74a-c7efd9061a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:48.132936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1700e5-58b7-4842-a74a-c7efd9061a6f admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:48.135665 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1700e5-58b7-4842-a74a-c7efd9061a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:48.136838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1700e5-58b7-4842-a74a-c7efd9061a6f admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:48.137679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1700e5-58b7-4842-a74a-c7efd9061a6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:48.145778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1700e5-58b7-4842-a74a-c7efd9061a6f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:48.206753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 840/1684] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:47 2022] POST /identity/v3/users => generated 285 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:48.208582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1700e5-58b7-4842-a74a-c7efd9061a6f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:48.223670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1700e5-58b7-4842-a74a-c7efd9061a6f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:48.238970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c1e6f-e190-4cca-8de9-cc9e8cb3ac8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:48.241563 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c1e6f-e190-4cca-8de9-cc9e8cb3ac8f admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:48.242218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c1e6f-e190-4cca-8de9-cc9e8cb3ac8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:48.242600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c1e6f-e190-4cca-8de9-cc9e8cb3ac8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:48.243452 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c1e6f-e190-4cca-8de9-cc9e8cb3ac8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:48.246491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c1e6f-e190-4cca-8de9-cc9e8cb3ac8f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:48.260688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:48.276008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 841/1685] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:48.277061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5cadf-181a-4ab7-82b1-eec75b1304be admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:48.278240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c1e6f-e190-4cca-8de9-cc9e8cb3ac8f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:48.293973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcbc2c-6710-4e5a-8242-7b89222f1ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:48.294895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcbc2c-6710-4e5a-8242-7b89222f1ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:48.295180 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcbc2c-6710-4e5a-8242-7b89222f1ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:48.295709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcbc2c-6710-4e5a-8242-7b89222f1ff3 admin admin] PATH_INFO: `/v3/projects/fb23e39ef1d34d27a3a3f074c40d5ce6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:48.296193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c1e6f-e190-4cca-8de9-cc9e8cb3ac8f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:48.296976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcbc2c-6710-4e5a-8242-7b89222f1ff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb23e39ef1d34d27a3a3f074c40d5ce6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:48.301204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcbc2c-6710-4e5a-8242-7b89222f1ff3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:48.303567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079cb13d-a947-45f2-8e07-c9ef1b3f5ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:48.304229 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cb13d-a947-45f2-8e07-c9ef1b3f5ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:48.304610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cb13d-a947-45f2-8e07-c9ef1b3f5ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:48.305002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cb13d-a947-45f2-8e07-c9ef1b3f5ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:48.305932 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079cb13d-a947-45f2-8e07-c9ef1b3f5ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:48.307878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079cb13d-a947-45f2-8e07-c9ef1b3f5ae2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:48.331187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 842/1686] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:48.335214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079cb13d-a947-45f2-8e07-c9ef1b3f5ae2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:48.348485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:48.359447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079cb13d-a947-45f2-8e07-c9ef1b3f5ae2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:48.374578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f05eb0-be0f-4fb7-a189-8677f1c938b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:48.382834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f05eb0-be0f-4fb7-a189-8677f1c938b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:48.384158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f05eb0-be0f-4fb7-a189-8677f1c938b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:48.385922 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f05eb0-be0f-4fb7-a189-8677f1c938b0 admin admin] PATH_INFO: `/v3/projects/e97557f7feb94e9dbbe237eeac7ad411/users/d176a2752df144dcbf93467fc49ecfa8/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:48.433136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f05eb0-be0f-4fb7-a189-8677f1c938b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e97557f7feb94e9dbbe237eeac7ad411, user_id=d176a2752df144dcbf93467fc49ecfa8, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:48.442406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f05eb0-be0f-4fb7-a189-8677f1c938b0 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:48.481669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 843/1687] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:48 2022] PUT /identity/v3/projects/e97557f7feb94e9dbbe237eeac7ad411/users/d176a2752df144dcbf93467fc49ecfa8/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:48.492270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c023a85-1097-4f51-8937-fd241649aa47 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:48.492559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c023a85-1097-4f51-8937-fd241649aa47 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:48.492921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c023a85-1097-4f51-8937-fd241649aa47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:48.493671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-affcbc2c-6710-4e5a-8242-7b89222f1ff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb23e39ef1d34d27a3a3f074c40d5ce6', 'request_id': 'req-affcbc2c-6710-4e5a-8242-7b89222f1ff3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:48.523076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-affcbc2c-6710-4e5a-8242-7b89222f1ff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb23e39ef1d34d27a3a3f074c40d5ce6', 'request_id': 'req-affcbc2c-6710-4e5a-8242-7b89222f1ff3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:48.553481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9c023a85-1097-4f51-8937-fd241649aa47 None None] MFA Rules not processed for user `d176a2752df144dcbf93467fc49ecfa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:48.555068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9c023a85-1097-4f51-8937-fd241649aa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:48.558522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-affcbc2c-6710-4e5a-8242-7b89222f1ff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb23e39ef1d34d27a3a3f074c40d5ce6', 'request_id': 'req-affcbc2c-6710-4e5a-8242-7b89222f1ff3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:48.605785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-affcbc2c-6710-4e5a-8242-7b89222f1ff3 admin admin] The token cache is being invalidate because project fb23e39ef1d34d27a3a3f074c40d5ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:17:48.606403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-affcbc2c-6710-4e5a-8242-7b89222f1ff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:48.607279 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-affcbc2c-6710-4e5a-8242-7b89222f1ff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:48.609539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 845/1688] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:17:48 2022] DELETE /identity/v3/projects/fb23e39ef1d34d27a3a3f074c40d5ce6 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:48.615253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288f79e-c319-4beb-a599-3fc7991222f3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:48.615743 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288f79e-c319-4beb-a599-3fc7991222f3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:48.616161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288f79e-c319-4beb-a599-3fc7991222f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:48.662085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d288f79e-c319-4beb-a599-3fc7991222f3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:48.670943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d288f79e-c319-4beb-a599-3fc7991222f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:48.717787 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 844/1689] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:17:48.720594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c023a85-1097-4f51-8937-fd241649aa47 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:48.725346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9c023a85-1097-4f51-8937-fd241649aa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:48.825303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 846/1690] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:48.827066 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c1234-a9d1-4c2f-baf7-37db19352c0b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:48.827378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c1234-a9d1-4c2f-baf7-37db19352c0b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:48.827897 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c1234-a9d1-4c2f-baf7-37db19352c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:48.857376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f99c1234-a9d1-4c2f-baf7-37db19352c0b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:48.861114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f99c1234-a9d1-4c2f-baf7-37db19352c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:48.888065 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 847/1691] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:17:48.892473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c1234-a9d1-4c2f-baf7-37db19352c0b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:48.914913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c023a85-1097-4f51-8937-fd241649aa47 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:48.928377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c81fd0-a9b5-4ef1-b1b4-d08c60135659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:48.929239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c1234-a9d1-4c2f-baf7-37db19352c0b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:48.929337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c81fd0-a9b5-4ef1-b1b4-d08c60135659 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:48.929470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c81fd0-a9b5-4ef1-b1b4-d08c60135659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:48.929835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c81fd0-a9b5-4ef1-b1b4-d08c60135659 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:48.930768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c81fd0-a9b5-4ef1-b1b4-d08c60135659 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:48.932490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c81fd0-a9b5-4ef1-b1b4-d08c60135659 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:48.949641 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111bbf1-29b1-4701-b98c-d0dad8675f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:48.950357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111bbf1-29b1-4701-b98c-d0dad8675f7e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:48.950645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111bbf1-29b1-4701-b98c-d0dad8675f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:48.950951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111bbf1-29b1-4701-b98c-d0dad8675f7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:48.951650 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2111bbf1-29b1-4701-b98c-d0dad8675f7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:17:48.952069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2111bbf1-29b1-4701-b98c-d0dad8675f7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:48.955971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2111bbf1-29b1-4701-b98c-d0dad8675f7e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:48.997750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 848/1692] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:17:49.003877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111bbf1-29b1-4701-b98c-d0dad8675f7e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:49.005805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 845/1693] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:48 2022] POST /identity/v3/projects => generated 369 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:49.009182 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c81fd0-a9b5-4ef1-b1b4-d08c60135659 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:49.037259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111bbf1-29b1-4701-b98c-d0dad8675f7e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:49.174908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:49.184849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c81fd0-a9b5-4ef1-b1b4-d08c60135659 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:49.201443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2d689-77da-48da-9205-6409f1f99756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:49.202164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2d689-77da-48da-9205-6409f1f99756 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:49.202523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2d689-77da-48da-9205-6409f1f99756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:49.202955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2d689-77da-48da-9205-6409f1f99756 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:49.204627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2d689-77da-48da-9205-6409f1f99756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:49.209767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2d689-77da-48da-9205-6409f1f99756 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:49.232012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64cc21-4521-44a1-9288-c855d3eebe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:49.233904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64cc21-4521-44a1-9288-c855d3eebe7b admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:49.234189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64cc21-4521-44a1-9288-c855d3eebe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:49.234461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64cc21-4521-44a1-9288-c855d3eebe7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:49.236259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64cc21-4521-44a1-9288-c855d3eebe7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:49.246853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64cc21-4521-44a1-9288-c855d3eebe7b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:49.251060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 846/1694] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:49 2022] POST /identity/v3/users => generated 291 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:49.255073 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2d689-77da-48da-9205-6409f1f99756 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:49.277982 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2d689-77da-48da-9205-6409f1f99756 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:49.293549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6459a44-d205-4c42-9a32-9790a3999537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:49.294290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6459a44-d205-4c42-9a32-9790a3999537 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:49.294578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6459a44-d205-4c42-9a32-9790a3999537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:49.296702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6459a44-d205-4c42-9a32-9790a3999537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:49.298401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6459a44-d205-4c42-9a32-9790a3999537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:49.299703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:49.307141 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6459a44-d205-4c42-9a32-9790a3999537 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:49.337919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 849/1695] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:49 2022] POST /identity/v3/projects => generated 365 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:49.342528 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64cc21-4521-44a1-9288-c855d3eebe7b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:49.352659 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 847/1696] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:17:49.354089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6459a44-d205-4c42-9a32-9790a3999537 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:49.393684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6459a44-d205-4c42-9a32-9790a3999537 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:49.531848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64cc21-4521-44a1-9288-c855d3eebe7b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:49.549155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe52ba-53fc-449b-ae22-88111b7fded7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:49.549892 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe52ba-53fc-449b-ae22-88111b7fded7 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:49.550149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe52ba-53fc-449b-ae22-88111b7fded7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:49.550948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe52ba-53fc-449b-ae22-88111b7fded7 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:49.551268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe52ba-53fc-449b-ae22-88111b7fded7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:49.553228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe52ba-53fc-449b-ae22-88111b7fded7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:49.563895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89006681-a04a-4a8d-a2cd-0a87962cfd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:49.565862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89006681-a04a-4a8d-a2cd-0a87962cfd71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:49.571748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89006681-a04a-4a8d-a2cd-0a87962cfd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:49.571748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89006681-a04a-4a8d-a2cd-0a87962cfd71 admin admin] PATH_INFO: `/v3/projects/2abf5c445d9347e89431ac7eb3ed8ca1/users/8c3d29abb13f431190b25480a963b270/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:49.594132 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89006681-a04a-4a8d-a2cd-0a87962cfd71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abf5c445d9347e89431ac7eb3ed8ca1, user_id=8c3d29abb13f431190b25480a963b270, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:49.599534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89006681-a04a-4a8d-a2cd-0a87962cfd71 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:49.612473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 850/1697] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:49 2022] POST /identity/v3/users => generated 283 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:49.614674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe52ba-53fc-449b-ae22-88111b7fded7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:49.629037 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe52ba-53fc-449b-ae22-88111b7fded7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:49.638438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 848/1698] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:49 2022] PUT /identity/v3/projects/2abf5c445d9347e89431ac7eb3ed8ca1/users/8c3d29abb13f431190b25480a963b270/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:49.640164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c922266-8f9b-4e49-ad7c-f080d6ec9737 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:49.640642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c922266-8f9b-4e49-ad7c-f080d6ec9737 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:49.641197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c922266-8f9b-4e49-ad7c-f080d6ec9737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:49.693606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5c922266-8f9b-4e49-ad7c-f080d6ec9737 None None] MFA Rules not processed for user `8da0a5a09a6041d68c5e445af1a492f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:49.698114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5c922266-8f9b-4e49-ad7c-f080d6ec9737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:49.814720 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18484458-900b-405f-a2fd-75c18af8f09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:49.817186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18484458-900b-405f-a2fd-75c18af8f09b admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:49.820721 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18484458-900b-405f-a2fd-75c18af8f09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:49.821416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18484458-900b-405f-a2fd-75c18af8f09b admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:49.822699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18484458-900b-405f-a2fd-75c18af8f09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:49.831473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18484458-900b-405f-a2fd-75c18af8f09b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:49.857984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 851/1699] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:49.857984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18484458-900b-405f-a2fd-75c18af8f09b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:49.880393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18484458-900b-405f-a2fd-75c18af8f09b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:49.883659 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 849/1700] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:49.884692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c922266-8f9b-4e49-ad7c-f080d6ec9737 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:49.895966 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8ffb7-6fc6-4562-888b-f62ba5fe5f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:49.897084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8ffb7-6fc6-4562-888b-f62ba5fe5f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:49.897084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8ffb7-6fc6-4562-888b-f62ba5fe5f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:49.897504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8ffb7-6fc6-4562-888b-f62ba5fe5f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:49.898396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8ffb7-6fc6-4562-888b-f62ba5fe5f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:49.900550 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8ffb7-6fc6-4562-888b-f62ba5fe5f6f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:49.916382 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c922266-8f9b-4e49-ad7c-f080d6ec9737 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:49.923157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 852/1701] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:49.924622 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8ffb7-6fc6-4562-888b-f62ba5fe5f6f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:49.929019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1db8ffb7-6fc6-4562-888b-f62ba5fe5f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:49.930968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948b554-904e-47c9-8dd7-540e06dd8692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:49.931662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948b554-904e-47c9-8dd7-540e06dd8692 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:49.932605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948b554-904e-47c9-8dd7-540e06dd8692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:49.933062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948b554-904e-47c9-8dd7-540e06dd8692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:49.934653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948b554-904e-47c9-8dd7-540e06dd8692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:49.936329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948b554-904e-47c9-8dd7-540e06dd8692 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:49.969135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 850/1702] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:49.974683 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948b554-904e-47c9-8dd7-540e06dd8692 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:49.987259 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948b554-904e-47c9-8dd7-540e06dd8692 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:49.997080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd2e1d-49ac-4675-9b87-9f3011c4ce1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:49.998645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd2e1d-49ac-4675-9b87-9f3011c4ce1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:49.998926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd2e1d-49ac-4675-9b87-9f3011c4ce1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:49.999197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd2e1d-49ac-4675-9b87-9f3011c4ce1c admin admin] PATH_INFO: `/v3/projects/2abf5c445d9347e89431ac7eb3ed8ca1/users/8c3d29abb13f431190b25480a963b270/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:50.003955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd2e1d-49ac-4675-9b87-9f3011c4ce1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abf5c445d9347e89431ac7eb3ed8ca1, user_id=8c3d29abb13f431190b25480a963b270, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:50.014134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd2e1d-49ac-4675-9b87-9f3011c4ce1c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:50.043465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 851/1703] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:49 2022] PUT /identity/v3/projects/2abf5c445d9347e89431ac7eb3ed8ca1/users/8c3d29abb13f431190b25480a963b270/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:50.044585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd2e1d-49ac-4675-9b87-9f3011c4ce1c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:50.107883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:50.108109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:50.119297 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8ffb7-6fc6-4562-888b-f62ba5fe5f6f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:50.207093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd2e1d-49ac-4675-9b87-9f3011c4ce1c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:50.218390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac8ea5-3941-4147-b002-8f12c98dbbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:50.221174 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac8ea5-3941-4147-b002-8f12c98dbbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:50.221315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead80f27-1bd8-4fe4-8232-34ac3540709e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:50.221469 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac8ea5-3941-4147-b002-8f12c98dbbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:50.221922 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac8ea5-3941-4147-b002-8f12c98dbbf2 admin admin] PATH_INFO: `/v3/projects/209166afa9024d94be6ed71b48cfac70/users/282f76da583e49c1ba2ca25bc62787fa/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:50.222023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead80f27-1bd8-4fe4-8232-34ac3540709e service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:50.222305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead80f27-1bd8-4fe4-8232-34ac3540709e service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:50.222583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead80f27-1bd8-4fe4-8232-34ac3540709e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:50.242051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac8ea5-3941-4147-b002-8f12c98dbbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209166afa9024d94be6ed71b48cfac70, user_id=282f76da583e49c1ba2ca25bc62787fa, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:50.245996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac8ea5-3941-4147-b002-8f12c98dbbf2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:50.267147 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 852/1704] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:50 2022] PUT /identity/v3/projects/209166afa9024d94be6ed71b48cfac70/users/282f76da583e49c1ba2ca25bc62787fa/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:50.269254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a44dc1-db8a-4954-8533-2d271feb2498 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:50.269547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a44dc1-db8a-4954-8533-2d271feb2498 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:50.270007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a44dc1-db8a-4954-8533-2d271feb2498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:50.293302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead80f27-1bd8-4fe4-8232-34ac3540709e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:50.294886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead80f27-1bd8-4fe4-8232-34ac3540709e service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:50.307345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 853/1705] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:50.308307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f2a44dc1-db8a-4954-8533-2d271feb2498 None None] MFA Rules not processed for user `8c3d29abb13f431190b25480a963b270`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:50.308823 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ff2b3-b529-47b7-8272-ca99bd6d78b5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:50.309115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ff2b3-b529-47b7-8272-ca99bd6d78b5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:50.309446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ff2b3-b529-47b7-8272-ca99bd6d78b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:50.309717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f2a44dc1-db8a-4954-8533-2d271feb2498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:50.388077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-584ff2b3-b529-47b7-8272-ca99bd6d78b5 None None] MFA Rules not processed for user `282f76da583e49c1ba2ca25bc62787fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:50.391665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-584ff2b3-b529-47b7-8272-ca99bd6d78b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:50.507627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 853/1706] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:50 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:50.527098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a44dc1-db8a-4954-8533-2d271feb2498 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:50.533691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f2a44dc1-db8a-4954-8533-2d271feb2498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:50.605350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 854/1707] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:50.606339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ff2b3-b529-47b7-8272-ca99bd6d78b5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:50.768992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a44dc1-db8a-4954-8533-2d271feb2498 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:50.785152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a5968-4c26-4e66-abee-3bf894b459e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:50.785975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a5968-4c26-4e66-abee-3bf894b459e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:50.788963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a5968-4c26-4e66-abee-3bf894b459e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:50.789654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:50.792918 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a5968-4c26-4e66-abee-3bf894b459e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:50.827785 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a5968-4c26-4e66-abee-3bf894b459e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:50.834562 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a5968-4c26-4e66-abee-3bf894b459e7 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:50.898648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 854/1708] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:50 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:50.921623 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ff2b3-b529-47b7-8272-ca99bd6d78b5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:50.931697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f21dad-56fd-4a31-b1c2-c38c4cd7e49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:50.932554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f21dad-56fd-4a31-b1c2-c38c4cd7e49d admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:50.933051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f21dad-56fd-4a31-b1c2-c38c4cd7e49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:50.934131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f21dad-56fd-4a31-b1c2-c38c4cd7e49d admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:50.934961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f21dad-56fd-4a31-b1c2-c38c4cd7e49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:50.937431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f21dad-56fd-4a31-b1c2-c38c4cd7e49d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:51.007290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 855/1709] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:17:50 2022] POST /identity/v3/projects => generated 365 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:51.010917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a5968-4c26-4e66-abee-3bf894b459e7 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:51.025103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a5968-4c26-4e66-abee-3bf894b459e7 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:51.053026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f21dad-56fd-4a31-b1c2-c38c4cd7e49d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:51.122788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:51.163550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de491d5-1afa-4890-8a6a-9f1e84359499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:51.164290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de491d5-1afa-4890-8a6a-9f1e84359499 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:51.164572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de491d5-1afa-4890-8a6a-9f1e84359499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:51.164895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de491d5-1afa-4890-8a6a-9f1e84359499 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:51.165779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de491d5-1afa-4890-8a6a-9f1e84359499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:51.168381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de491d5-1afa-4890-8a6a-9f1e84359499 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:51.207330 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 855/1710] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:17:51 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:51.213508 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de491d5-1afa-4890-8a6a-9f1e84359499 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:51.245198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de491d5-1afa-4890-8a6a-9f1e84359499 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:51.261071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58d0dd-5a4b-426e-ba3d-dae11e23f145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:51.262404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f21dad-56fd-4a31-b1c2-c38c4cd7e49d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:51.263450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58d0dd-5a4b-426e-ba3d-dae11e23f145 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:51.263910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58d0dd-5a4b-426e-ba3d-dae11e23f145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:51.264195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58d0dd-5a4b-426e-ba3d-dae11e23f145 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:51.266176 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58d0dd-5a4b-426e-ba3d-dae11e23f145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:51.273633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58d0dd-5a4b-426e-ba3d-dae11e23f145 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:51.273812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7a998-155d-4b7f-a2e5-b4fbf7e029f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:51.273812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7a998-155d-4b7f-a2e5-b4fbf7e029f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:51.273812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7a998-155d-4b7f-a2e5-b4fbf7e029f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:51.275210 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7a998-155d-4b7f-a2e5-b4fbf7e029f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:51.300918 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 856/1711] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:51.302898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58d0dd-5a4b-426e-ba3d-dae11e23f145 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:51.343525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58d0dd-5a4b-426e-ba3d-dae11e23f145 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:51.353505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0346f4-684a-4cb1-9b1a-96cb49812e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:51.369834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0346f4-684a-4cb1-9b1a-96cb49812e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:51.370356 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0346f4-684a-4cb1-9b1a-96cb49812e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:51.370780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0346f4-684a-4cb1-9b1a-96cb49812e81 admin admin] PATH_INFO: `/v3/projects/04c4cb877f884aceac62eb9234d8d262/users/a7e111a4901d466f8c9b856973c7e199/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:51.412864 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0346f4-684a-4cb1-9b1a-96cb49812e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c4cb877f884aceac62eb9234d8d262, user_id=a7e111a4901d466f8c9b856973c7e199, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:51.415589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0346f4-684a-4cb1-9b1a-96cb49812e81 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:51.442542 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 857/1712] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:51 2022] PUT /identity/v3/projects/04c4cb877f884aceac62eb9234d8d262/users/a7e111a4901d466f8c9b856973c7e199/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:51.445531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c6d5a-1b5a-438f-9a2c-78653a7c8def None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:51.446653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c6d5a-1b5a-438f-9a2c-78653a7c8def None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:51.446940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c6d5a-1b5a-438f-9a2c-78653a7c8def None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:51.452117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 858/1713] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:17:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:17:51.452950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c6d5a-1b5a-438f-9a2c-78653a7c8def None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:51.481392 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7a998-155d-4b7f-a2e5-b4fbf7e029f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:51.482978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7a998-155d-4b7f-a2e5-b4fbf7e029f7 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:51.511844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 856/1714] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:51 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:51.537618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20be1cd-fd09-45ea-8519-b9c9519a4801 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:51.538004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20be1cd-fd09-45ea-8519-b9c9519a4801 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:51.538291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20be1cd-fd09-45ea-8519-b9c9519a4801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:51.575731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c20be1cd-fd09-45ea-8519-b9c9519a4801 None None] MFA Rules not processed for user `7ee40bcb7905490c9ad3e39e7a93b122`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:51.576830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c20be1cd-fd09-45ea-8519-b9c9519a4801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:51.669219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c6d5a-1b5a-438f-9a2c-78653a7c8def None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:51.688782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5e0fe-31b7-4138-90d6-0b5e44100a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:51.690195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5e0fe-31b7-4138-90d6-0b5e44100a43 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:51.690514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5e0fe-31b7-4138-90d6-0b5e44100a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:51.690968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5e0fe-31b7-4138-90d6-0b5e44100a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:51.691852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5e0fe-31b7-4138-90d6-0b5e44100a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:51.693998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5e0fe-31b7-4138-90d6-0b5e44100a43 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:51.712190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 859/1715] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:51.714129 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab423fe-1983-472f-97ab-59435a1347a9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:51.714426 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab423fe-1983-472f-97ab-59435a1347a9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:51.714888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab423fe-1983-472f-97ab-59435a1347a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:51.747624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 857/1716] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:17:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:17:51.748537 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20be1cd-fd09-45ea-8519-b9c9519a4801 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:51.758851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-7ab423fe-1983-472f-97ab-59435a1347a9 None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:51.760421 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7ab423fe-1983-472f-97ab-59435a1347a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:51.777888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20be1cd-fd09-45ea-8519-b9c9519a4801 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:51.792422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdb6a0-71cf-42c8-86c2-7255ec4106ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:51.793246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdb6a0-71cf-42c8-86c2-7255ec4106ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:51.793510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdb6a0-71cf-42c8-86c2-7255ec4106ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:51.793813 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdb6a0-71cf-42c8-86c2-7255ec4106ac admin admin] PATH_INFO: `/v3/projects/04c4cb877f884aceac62eb9234d8d262/users/a7e111a4901d466f8c9b856973c7e199/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:51.796563 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdb6a0-71cf-42c8-86c2-7255ec4106ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c4cb877f884aceac62eb9234d8d262, user_id=a7e111a4901d466f8c9b856973c7e199, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:51.800572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdb6a0-71cf-42c8-86c2-7255ec4106ac admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:51.807002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:51.849341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 858/1717] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:17:51 2022] PUT /identity/v3/projects/04c4cb877f884aceac62eb9234d8d262/users/a7e111a4901d466f8c9b856973c7e199/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:51.849341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdb6a0-71cf-42c8-86c2-7255ec4106ac admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:51.855906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8ffdb6a0-71cf-42c8-86c2-7255ec4106ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:51.886226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 860/1718] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:17:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:51.889453 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4df983-db8e-4000-bdb4-8548003ee1ca None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:51.889757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4df983-db8e-4000-bdb4-8548003ee1ca None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:51.894613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4df983-db8e-4000-bdb4-8548003ee1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:51.970585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5a4df983-db8e-4000-bdb4-8548003ee1ca None None] MFA Rules not processed for user `a7e111a4901d466f8c9b856973c7e199`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:51.976826 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5a4df983-db8e-4000-bdb4-8548003ee1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:52.029854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:52.108894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdb6a0-71cf-42c8-86c2-7255ec4106ac admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:52.126080 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb3fc2-5b76-4c06-820c-d988ce645634 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:52.128499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb3fc2-5b76-4c06-820c-d988ce645634 service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:52.129722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb3fc2-5b76-4c06-820c-d988ce645634 service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:52.130248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb3fc2-5b76-4c06-820c-d988ce645634 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:52.169475 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 861/1719] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:52.173117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4df983-db8e-4000-bdb4-8548003ee1ca None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:52.177067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb3fc2-5b76-4c06-820c-d988ce645634 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:52.177275 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5a4df983-db8e-4000-bdb4-8548003ee1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:52.179240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb3fc2-5b76-4c06-820c-d988ce645634 service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:52.197810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 859/1720] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:52.212566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb3fc2-5b76-4c06-820c-d988ce645634 service placement] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:52.220333 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-84fb3fc2-5b76-4c06-820c-d988ce645634 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:52.363088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4df983-db8e-4000-bdb4-8548003ee1ca None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:52.374098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7f873-cf49-4380-8ddf-e798e99f3e0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:52.374917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f873-cf49-4380-8ddf-e798e99f3e0a service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:52.375079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f873-cf49-4380-8ddf-e798e99f3e0a service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:52.375352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f873-cf49-4380-8ddf-e798e99f3e0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:52.511922 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb3fc2-5b76-4c06-820c-d988ce645634 service placement] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:52.526640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32aedc-ae7c-44ff-9b2e-175cc75e9591 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:52.527316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32aedc-ae7c-44ff-9b2e-175cc75e9591 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:52.527600 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32aedc-ae7c-44ff-9b2e-175cc75e9591 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:52.527886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32aedc-ae7c-44ff-9b2e-175cc75e9591 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:52.563354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7f873-cf49-4380-8ddf-e798e99f3e0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:52.567086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7f873-cf49-4380-8ddf-e798e99f3e0a service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:52.568793 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32aedc-ae7c-44ff-9b2e-175cc75e9591 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:52.571588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32aedc-ae7c-44ff-9b2e-175cc75e9591 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:52.590787 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 862/1721] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:17:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:17:52.638426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 860/1722] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:52 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:52.920070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7f873-cf49-4380-8ddf-e798e99f3e0a service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:52.936532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7f873-cf49-4380-8ddf-e798e99f3e0a service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:52.959315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aefe58-e482-4bb3-83d6-3b6190aa656b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:52.960156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aefe58-e482-4bb3-83d6-3b6190aa656b service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:52.960449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aefe58-e482-4bb3-83d6-3b6190aa656b service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:52.960731 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aefe58-e482-4bb3-83d6-3b6190aa656b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:53.098328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:53.105871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aefe58-e482-4bb3-83d6-3b6190aa656b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:53.107743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aefe58-e482-4bb3-83d6-3b6190aa656b service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:53.144179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 863/1723] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:55.650792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32aedc-ae7c-44ff-9b2e-175cc75e9591 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:55.652114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4f32aedc-ae7c-44ff-9b2e-175cc75e9591 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:55.911227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32aedc-ae7c-44ff-9b2e-175cc75e9591 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:55.933973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e71a78-bdd2-4e69-9b03-ab3844be0fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:55.934866 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e71a78-bdd2-4e69-9b03-ab3844be0fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:55.935156 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e71a78-bdd2-4e69-9b03-ab3844be0fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:55.935413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e71a78-bdd2-4e69-9b03-ab3844be0fc6 admin admin] PATH_INFO: `/v3/users/96a0ff77014d41beb98e5748f8f255a8` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:55.939131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e71a78-bdd2-4e69-9b03-ab3844be0fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96a0ff77014d41beb98e5748f8f255a8)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:55.949371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e71a78-bdd2-4e69-9b03-ab3844be0fc6 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:56.051264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:INSERT=1,keystone:SELECT=94,keystone:DELETE=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:56.092048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-93e71a78-bdd2-4e69-9b03-ab3844be0fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96a0ff77014d41beb98e5748f8f255a8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:56.110860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-93e71a78-bdd2-4e69-9b03-ab3844be0fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96a0ff77014d41beb98e5748f8f255a8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:56.133149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-93e71a78-bdd2-4e69-9b03-ab3844be0fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96a0ff77014d41beb98e5748f8f255a8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:56.133909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-93e71a78-bdd2-4e69-9b03-ab3844be0fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96a0ff77014d41beb98e5748f8f255a8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:56.142066 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-93e71a78-bdd2-4e69-9b03-ab3844be0fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96a0ff77014d41beb98e5748f8f255a8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:56.150429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 861/1724] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:17:55 2022] DELETE /identity/v3/users/96a0ff77014d41beb98e5748f8f255a8 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:56.391875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b41529-2eec-4e91-8bb8-22cf57503c05 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:56.392971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b41529-2eec-4e91-8bb8-22cf57503c05 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:56.392971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b41529-2eec-4e91-8bb8-22cf57503c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:56.516808 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-34b41529-2eec-4e91-8bb8-22cf57503c05 None None] MFA Rules not processed for user `a20f6932d763461ea26c2d60ed58a59f`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:56.518549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-34b41529-2eec-4e91-8bb8-22cf57503c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:56.650207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dbe9e-f73d-4a59-9d85-56429ea784b8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:56.650610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dbe9e-f73d-4a59-9d85-56429ea784b8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:56.650982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dbe9e-f73d-4a59-9d85-56429ea784b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:56.716780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9c4dbe9e-f73d-4a59-9d85-56429ea784b8 None None] MFA Rules not processed for user `f668c4c6745e4f1eae20d8a3a12cf0cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:56.719357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9c4dbe9e-f73d-4a59-9d85-56429ea784b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:56.794426 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 864/1725] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:17:56.807430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192d1cc-2749-462d-9c6d-0904e6c4fbfb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:56.808423 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192d1cc-2749-462d-9c6d-0904e6c4fbfb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:56.809233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192d1cc-2749-462d-9c6d-0904e6c4fbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:56.843698 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4192d1cc-2749-462d-9c6d-0904e6c4fbfb None None] MFA Rules not processed for user `8da0a5a09a6041d68c5e445af1a492f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:56.845389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4192d1cc-2749-462d-9c6d-0904e6c4fbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:56.939590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 865/1726] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:56.941058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192d1cc-2749-462d-9c6d-0904e6c4fbfb None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:56.943012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4192d1cc-2749-462d-9c6d-0904e6c4fbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:57.076181 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192d1cc-2749-462d-9c6d-0904e6c4fbfb None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:57.088788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:57.091300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:57.091704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:57.092048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] PATH_INFO: `/v3/projects/7540c93de42e46ae9c4c02ae1c20032d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:57.098461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7540c93de42e46ae9c4c02ae1c20032d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:57.125122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:57.218324 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 862/1727] 10.222.0.42 () {58 vars in 1015 bytes} [Thu May 26 18:17:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 569 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:17:57.219628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dbe9e-f73d-4a59-9d85-56429ea784b8 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:57.221738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9c4dbe9e-f73d-4a59-9d85-56429ea784b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:57.313931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:57.375079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7540c93de42e46ae9c4c02ae1c20032d', 'request_id': 'req-06065e98-051a-469e-93f0-4e6d5c598da5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:57.400747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7540c93de42e46ae9c4c02ae1c20032d', 'request_id': 'req-06065e98-051a-469e-93f0-4e6d5c598da5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:57.442098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7540c93de42e46ae9c4c02ae1c20032d', 'request_id': 'req-06065e98-051a-469e-93f0-4e6d5c598da5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:57.488639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] The token cache is being invalidate because project 7540c93de42e46ae9c4c02ae1c20032d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:17:57.489010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:57.491173 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:57.498437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 866/1728] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:17:56 2022] DELETE /identity/v3/projects/7540c93de42e46ae9c4c02ae1c20032d => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:57.499893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:57.504898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:57.527629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dbe9e-f73d-4a59-9d85-56429ea784b8 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:57.689005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:57.756237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef8908-62ef-4b5d-a37e-85d0dc018094 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:57.757295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef8908-62ef-4b5d-a37e-85d0dc018094 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:57.757873 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef8908-62ef-4b5d-a37e-85d0dc018094 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:57.758353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef8908-62ef-4b5d-a37e-85d0dc018094 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:57.760408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e1ef8908-62ef-4b5d-a37e-85d0dc018094 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:57.778888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06065e98-051a-469e-93f0-4e6d5c598da5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:57.824874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:57.825806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:57.826117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:57.826466 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] PATH_INFO: `/v3/users/d176a2752df144dcbf93467fc49ecfa8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:57.838908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d176a2752df144dcbf93467fc49ecfa8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:57.840956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:57.900301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef8908-62ef-4b5d-a37e-85d0dc018094 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:57.904610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef8908-62ef-4b5d-a37e-85d0dc018094 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:57.930835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 863/1729] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:57.932107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef8908-62ef-4b5d-a37e-85d0dc018094 service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:57.933919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e1ef8908-62ef-4b5d-a37e-85d0dc018094 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:58.009282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd176a2752df144dcbf93467fc49ecfa8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.027086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd176a2752df144dcbf93467fc49ecfa8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.027807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd176a2752df144dcbf93467fc49ecfa8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.029258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd176a2752df144dcbf93467fc49ecfa8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.104430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd176a2752df144dcbf93467fc49ecfa8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.125506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 867/1730] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:17:57 2022] DELETE /identity/v3/users/d176a2752df144dcbf93467fc49ecfa8 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:58.126456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef8908-62ef-4b5d-a37e-85d0dc018094 service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:58.127592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:58.129507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:58.234652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:58.296464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a43c99-0a3c-4e72-a8fb-560dc64579c4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:58.300100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713499e-0ba2-45e3-ba3b-4ac601bc1f11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:58.301146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713499e-0ba2-45e3-ba3b-4ac601bc1f11 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:58.303424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713499e-0ba2-45e3-ba3b-4ac601bc1f11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:58.303770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713499e-0ba2-45e3-ba3b-4ac601bc1f11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:58.306557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5713499e-0ba2-45e3-ba3b-4ac601bc1f11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:58.310054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64606f96-7bbd-45e4-8dfe-2e222092a6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:58.311232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64606f96-7bbd-45e4-8dfe-2e222092a6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:58.311866 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64606f96-7bbd-45e4-8dfe-2e222092a6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:58.312417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64606f96-7bbd-45e4-8dfe-2e222092a6f4 admin admin] PATH_INFO: `/v3/users/896add8d2ac54725a350b86ac87897a5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:58.332555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64606f96-7bbd-45e4-8dfe-2e222092a6f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=896add8d2ac54725a350b86ac87897a5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:58.342358 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64606f96-7bbd-45e4-8dfe-2e222092a6f4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:58.352813 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713499e-0ba2-45e3-ba3b-4ac601bc1f11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:58.358617 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713499e-0ba2-45e3-ba3b-4ac601bc1f11 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:58.436371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 864/1731] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:58.437429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713499e-0ba2-45e3-ba3b-4ac601bc1f11 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:58.448211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5713499e-0ba2-45e3-ba3b-4ac601bc1f11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:58.466017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64606f96-7bbd-45e4-8dfe-2e222092a6f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '896add8d2ac54725a350b86ac87897a5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.485709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713499e-0ba2-45e3-ba3b-4ac601bc1f11 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:58.490259 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64606f96-7bbd-45e4-8dfe-2e222092a6f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '896add8d2ac54725a350b86ac87897a5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.491099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64606f96-7bbd-45e4-8dfe-2e222092a6f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '896add8d2ac54725a350b86ac87897a5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.491775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64606f96-7bbd-45e4-8dfe-2e222092a6f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '896add8d2ac54725a350b86ac87897a5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.515327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd36d2-5c3e-4e27-acbb-9eeb84008185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:58.520448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd36d2-5c3e-4e27-acbb-9eeb84008185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:58.521282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd36d2-5c3e-4e27-acbb-9eeb84008185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:58.521820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd36d2-5c3e-4e27-acbb-9eeb84008185 admin admin] PATH_INFO: `/v3/users/282f76da583e49c1ba2ca25bc62787fa` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:58.526894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd36d2-5c3e-4e27-acbb-9eeb84008185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=282f76da583e49c1ba2ca25bc62787fa)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:58.528974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd36d2-5c3e-4e27-acbb-9eeb84008185 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:58.531312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64606f96-7bbd-45e4-8dfe-2e222092a6f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '896add8d2ac54725a350b86ac87897a5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.576847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 868/1732] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:17:58 2022] DELETE /identity/v3/users/896add8d2ac54725a350b86ac87897a5 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:58.578652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58d30-7e17-4643-864d-0863425dcfd4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:58.578948 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58d30-7e17-4643-864d-0863425dcfd4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:58.579247 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58d30-7e17-4643-864d-0863425dcfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:58.653978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4bb58d30-7e17-4643-864d-0863425dcfd4 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:58.657807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4bb58d30-7e17-4643-864d-0863425dcfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:58.678211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:58.695328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d1dd36d2-5c3e-4e27-acbb-9eeb84008185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '282f76da583e49c1ba2ca25bc62787fa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.717327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d1dd36d2-5c3e-4e27-acbb-9eeb84008185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '282f76da583e49c1ba2ca25bc62787fa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.766369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d1dd36d2-5c3e-4e27-acbb-9eeb84008185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '282f76da583e49c1ba2ca25bc62787fa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.768028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d1dd36d2-5c3e-4e27-acbb-9eeb84008185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '282f76da583e49c1ba2ca25bc62787fa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.791775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d1dd36d2-5c3e-4e27-acbb-9eeb84008185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '282f76da583e49c1ba2ca25bc62787fa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:58.819985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 865/1733] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:17:58 2022] DELETE /identity/v3/users/282f76da583e49c1ba2ca25bc62787fa => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:58.821553 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519dd44-c29f-4589-a7f4-726c0e051602 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:58.827085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519dd44-c29f-4589-a7f4-726c0e051602 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:58.827529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519dd44-c29f-4589-a7f4-726c0e051602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:58.925366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 869/1734] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:58.926995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b7f3a-a2f8-453e-92f7-6cb2ad233c2c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:58.932981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b7f3a-a2f8-453e-92f7-6cb2ad233c2c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:58.933302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b7f3a-a2f8-453e-92f7-6cb2ad233c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:58.947121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f519dd44-c29f-4589-a7f4-726c0e051602 None None] MFA Rules not processed for user `3b3f8de4a3b54d58806506b827cfa89a`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:58.948418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f519dd44-c29f-4589-a7f4-726c0e051602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:59.028161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-609b7f3a-a2f8-453e-92f7-6cb2ad233c2c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:59.037682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-609b7f3a-a2f8-453e-92f7-6cb2ad233c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:59.212401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 866/1735] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:59.212401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519dd44-c29f-4589-a7f4-726c0e051602 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:59.216088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f519dd44-c29f-4589-a7f4-726c0e051602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:59.313310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:59.352509 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 870/1736] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:59.354157 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957d5c6-66a5-410b-a639-98defe8f164e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:59.354447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957d5c6-66a5-410b-a639-98defe8f164e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:59.354722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957d5c6-66a5-410b-a639-98defe8f164e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:59.422809 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1957d5c6-66a5-410b-a639-98defe8f164e None None] MFA Rules not processed for user `3b3f8de4a3b54d58806506b827cfa89a`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:59.425632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1957d5c6-66a5-410b-a639-98defe8f164e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:59.467849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 871/1737] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:17:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:59.489077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957d5c6-66a5-410b-a639-98defe8f164e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:59.493092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1957d5c6-66a5-410b-a639-98defe8f164e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:59.524466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519dd44-c29f-4589-a7f4-726c0e051602 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:59.533051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f18bd-6b93-4ce4-ab67-a21144f7c1e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:59.535936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f18bd-6b93-4ce4-ab67-a21144f7c1e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:59.536186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f18bd-6b93-4ce4-ab67-a21144f7c1e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:59.536465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f18bd-6b93-4ce4-ab67-a21144f7c1e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:59.558599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f18bd-6b93-4ce4-ab67-a21144f7c1e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:59.560060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f18bd-6b93-4ce4-ab67-a21144f7c1e8 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:59.585502 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 867/1738] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:59.743063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957d5c6-66a5-410b-a639-98defe8f164e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:59.753652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b84d8d-cf78-4735-a241-1246b1486b9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:59.754546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b84d8d-cf78-4735-a241-1246b1486b9b service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:59.754931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b84d8d-cf78-4735-a241-1246b1486b9b service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:59.755410 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b84d8d-cf78-4735-a241-1246b1486b9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:59.775398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b84d8d-cf78-4735-a241-1246b1486b9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:59.778069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b84d8d-cf78-4735-a241-1246b1486b9b service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:59.795348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 872/1739] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:59.848085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f18bd-6b93-4ce4-ab67-a21144f7c1e8 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:59.850240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-299f18bd-6b93-4ce4-ab67-a21144f7c1e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:59.878119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f18bd-6b93-4ce4-ab67-a21144f7c1e8 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:59.897292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0230b-0ab4-44bf-9353-9d9183094638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:59.900370 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0230b-0ab4-44bf-9353-9d9183094638 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:59.902013 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0230b-0ab4-44bf-9353-9d9183094638 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:59.903631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0230b-0ab4-44bf-9353-9d9183094638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:59.973251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0230b-0ab4-44bf-9353-9d9183094638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:59.979748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0230b-0ab4-44bf-9353-9d9183094638 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:00.055480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 868/1740] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:17:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:01.084637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b84d8d-cf78-4735-a241-1246b1486b9b service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:01.086342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-41b84d8d-cf78-4735-a241-1246b1486b9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:01.132071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b84d8d-cf78-4735-a241-1246b1486b9b service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:01.158767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:01.160983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:01.162894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:01.163232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985 admin admin] PATH_INFO: `/v3/projects/9283da8d71e843b6a92d56858470ddb2` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:01.166459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9283da8d71e843b6a92d56858470ddb2)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:01.191767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:01.396183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9283da8d71e843b6a92d56858470ddb2', 'request_id': 'req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:01.416927 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9283da8d71e843b6a92d56858470ddb2', 'request_id': 'req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:01.438813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9283da8d71e843b6a92d56858470ddb2', 'request_id': 'req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:01.475700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985 admin admin] The token cache is being invalidate because project 9283da8d71e843b6a92d56858470ddb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:01.476438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:01.477384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb6e35e4-c75f-4b3c-a442-a9fb2ebeb985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:01.479714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 873/1741] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:01 2022] DELETE /identity/v3/projects/9283da8d71e843b6a92d56858470ddb2 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:01.488723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4323554-856e-425e-a8c8-bfff453b91c3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:01.489633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4323554-856e-425e-a8c8-bfff453b91c3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:01.490161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4323554-856e-425e-a8c8-bfff453b91c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:01.533062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f4323554-856e-425e-a8c8-bfff453b91c3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:01.540969 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f4323554-856e-425e-a8c8-bfff453b91c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:01.623234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7faf6a1-d63c-4b3d-a662-df2bc1b9dabc None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:01.623844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7faf6a1-d63c-4b3d-a662-df2bc1b9dabc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:01.624356 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7faf6a1-d63c-4b3d-a662-df2bc1b9dabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:01.680029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e7faf6a1-d63c-4b3d-a662-df2bc1b9dabc None None] MFA Rules not processed for user `f668c4c6745e4f1eae20d8a3a12cf0cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:01.681923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e7faf6a1-d63c-4b3d-a662-df2bc1b9dabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:01.706765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:01.818276 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 869/1742] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:01.825175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb055-bab6-460b-8214-ad383e621448 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:01.825499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb055-bab6-460b-8214-ad383e621448 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:01.825857 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb055-bab6-460b-8214-ad383e621448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:01.868893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-917cb055-bab6-460b-8214-ad383e621448 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:01.872370 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-917cb055-bab6-460b-8214-ad383e621448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:01.905454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 874/1744] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:18:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:01.907274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7faf6a1-d63c-4b3d-a662-df2bc1b9dabc None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:01.913023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 870/1744] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:01.933210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7faf6a1-d63c-4b3d-a662-df2bc1b9dabc None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:01.958608 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7dece-caf4-4769-8d47-a8b370922813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:01.959727 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7dece-caf4-4769-8d47-a8b370922813 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:01.959727 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7dece-caf4-4769-8d47-a8b370922813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:01.959950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7dece-caf4-4769-8d47-a8b370922813 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:01.962080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7dece-caf4-4769-8d47-a8b370922813 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:01.962474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7dece-caf4-4769-8d47-a8b370922813 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:01.965285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7dece-caf4-4769-8d47-a8b370922813 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:01.975167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:02.015712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cb055-bab6-460b-8214-ad383e621448 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:02.018173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-917cb055-bab6-460b-8214-ad383e621448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:02.023560 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 875/1745] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:02.029178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7dece-caf4-4769-8d47-a8b370922813 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:02.049918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cb055-bab6-460b-8214-ad383e621448 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:02.062896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7dece-caf4-4769-8d47-a8b370922813 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:02.068747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92fa21-ecf8-4420-b865-3328494d2882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:02.069878 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92fa21-ecf8-4420-b865-3328494d2882 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:02.071168 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92fa21-ecf8-4420-b865-3328494d2882 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:02.071560 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92fa21-ecf8-4420-b865-3328494d2882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:02.077095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93955c-f1bf-4910-b349-542b179d9c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:02.077908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93955c-f1bf-4910-b349-542b179d9c81 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:02.078577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93955c-f1bf-4910-b349-542b179d9c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:02.078870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93955c-f1bf-4910-b349-542b179d9c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:02.080774 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93955c-f1bf-4910-b349-542b179d9c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:02.084356 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93955c-f1bf-4910-b349-542b179d9c81 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:02.106828 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92fa21-ecf8-4420-b865-3328494d2882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:02.108490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92fa21-ecf8-4420-b865-3328494d2882 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:02.158917 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 871/1746] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:02.161886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92fa21-ecf8-4420-b865-3328494d2882 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:02.169304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-be92fa21-ecf8-4420-b865-3328494d2882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:02.169584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 876/1747] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:02 2022] POST /identity/v3/projects => generated 371 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:02.174813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93955c-f1bf-4910-b349-542b179d9c81 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:02.238524 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92fa21-ecf8-4420-b865-3328494d2882 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:02.416507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93955c-f1bf-4910-b349-542b179d9c81 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:02.434220 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdc061-2168-40b1-aed9-f7125bb98629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:02.437851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc061-2168-40b1-aed9-f7125bb98629 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:02.439077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc061-2168-40b1-aed9-f7125bb98629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:02.441301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc061-2168-40b1-aed9-f7125bb98629 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:02.442108 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdc061-2168-40b1-aed9-f7125bb98629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:02.447673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdc061-2168-40b1-aed9-f7125bb98629 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:02.475209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:02.483253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:02.485637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:02.486231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] PATH_INFO: `/v3/projects/e97557f7feb94e9dbbe237eeac7ad411` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:02.493018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] RBAC: Authorizing `identity:delete_project(project_id=e97557f7feb94e9dbbe237eeac7ad411)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:02.500811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:02.506603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 877/1748] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:02 2022] POST /identity/v3/users => generated 286 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:02.523551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdc061-2168-40b1-aed9-f7125bb98629 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:02.549398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdc061-2168-40b1-aed9-f7125bb98629 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:02.578890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a39fa-16cf-4548-982e-c11547da842d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:02.579590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a39fa-16cf-4548-982e-c11547da842d admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:02.585964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a39fa-16cf-4548-982e-c11547da842d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:02.586290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a39fa-16cf-4548-982e-c11547da842d admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:02.587079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a39fa-16cf-4548-982e-c11547da842d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:02.588744 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a39fa-16cf-4548-982e-c11547da842d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:02.631838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 878/1749] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:02.633461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a39fa-16cf-4548-982e-c11547da842d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:02.645201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e97557f7feb94e9dbbe237eeac7ad411', 'request_id': 'req-068596c7-c26b-4cf2-a951-3360e7019fef'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:02.655681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a39fa-16cf-4548-982e-c11547da842d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:02.672141 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e97557f7feb94e9dbbe237eeac7ad411', 'request_id': 'req-068596c7-c26b-4cf2-a951-3360e7019fef'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:02.681071 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e97557f7feb94e9dbbe237eeac7ad411', 'request_id': 'req-068596c7-c26b-4cf2-a951-3360e7019fef'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:02.698709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] The token cache is being invalidate because project e97557f7feb94e9dbbe237eeac7ad411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:02.699032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:02.710834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:02.715715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 872/1750] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:02 2022] DELETE /identity/v3/projects/e97557f7feb94e9dbbe237eeac7ad411 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:02.721932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:02.722115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:02.863912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00c3b1-0499-404c-8f55-0663b987bece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:02.866748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00c3b1-0499-404c-8f55-0663b987bece admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:02.867080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00c3b1-0499-404c-8f55-0663b987bece admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:02.867368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00c3b1-0499-404c-8f55-0663b987bece admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:02.869123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00c3b1-0499-404c-8f55-0663b987bece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:02.874513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00c3b1-0499-404c-8f55-0663b987bece admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:02.887157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:02.918337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068596c7-c26b-4cf2-a951-3360e7019fef admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:02.919884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 879/1751] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:02.924745 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00c3b1-0499-404c-8f55-0663b987bece admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:02.932704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8f00c3b1-0499-404c-8f55-0663b987bece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:02.943576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a520d-72aa-430b-8c46-1129f2f90a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:02.944600 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a520d-72aa-430b-8c46-1129f2f90a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:02.944896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a520d-72aa-430b-8c46-1129f2f90a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:02.945178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a520d-72aa-430b-8c46-1129f2f90a2b admin admin] PATH_INFO: `/v3/users/8c3d29abb13f431190b25480a963b270` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:02.948544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a520d-72aa-430b-8c46-1129f2f90a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c3d29abb13f431190b25480a963b270)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:02.959973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a520d-72aa-430b-8c46-1129f2f90a2b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:02.985656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00c3b1-0499-404c-8f55-0663b987bece admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:03.017862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15227f-709d-4555-8acc-22d8a3139b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:03.018796 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15227f-709d-4555-8acc-22d8a3139b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:03.019078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15227f-709d-4555-8acc-22d8a3139b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:03.019354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15227f-709d-4555-8acc-22d8a3139b99 admin admin] PATH_INFO: `/v3/projects/8c9683df298148b1a0fa8bb446fb7e45/users/d7d44386436c4dbfa66aeeb5a9b04a71/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:03.105459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15227f-709d-4555-8acc-22d8a3139b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c9683df298148b1a0fa8bb446fb7e45, user_id=d7d44386436c4dbfa66aeeb5a9b04a71, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:03.117869 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15227f-709d-4555-8acc-22d8a3139b99 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:03.175229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a78a520d-72aa-430b-8c46-1129f2f90a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c3d29abb13f431190b25480a963b270'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:03.185841 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 880/1752] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:02 2022] PUT /identity/v3/projects/8c9683df298148b1a0fa8bb446fb7e45/users/d7d44386436c4dbfa66aeeb5a9b04a71/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:03.185841 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32ae56-2cc5-4fa3-8e6d-29cf2f4693e2 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:03.185841 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32ae56-2cc5-4fa3-8e6d-29cf2f4693e2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:03.185841 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32ae56-2cc5-4fa3-8e6d-29cf2f4693e2 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:03.198628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 881/1753] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:18:03 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:18:03.200198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3185d-7aa8-4d2a-9a8a-c159f8d5f6e7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:03.200756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3185d-7aa8-4d2a-9a8a-c159f8d5f6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:03.200877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3185d-7aa8-4d2a-9a8a-c159f8d5f6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:03.210265 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a78a520d-72aa-430b-8c46-1129f2f90a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c3d29abb13f431190b25480a963b270'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:03.294881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a78a520d-72aa-430b-8c46-1129f2f90a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c3d29abb13f431190b25480a963b270'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:03.303414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a78a520d-72aa-430b-8c46-1129f2f90a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c3d29abb13f431190b25480a963b270'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:03.364828 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a78a520d-72aa-430b-8c46-1129f2f90a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c3d29abb13f431190b25480a963b270'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:03.390309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 873/1754] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:02 2022] DELETE /identity/v3/users/8c3d29abb13f431190b25480a963b270 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:03.392079 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b9f65-59bb-4433-812b-8ffa5b058328 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:03.398781 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b9f65-59bb-4433-812b-8ffa5b058328 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:03.399137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b9f65-59bb-4433-812b-8ffa5b058328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:03.404139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-67d3185d-7aa8-4d2a-9a8a-c159f8d5f6e7 None None] MFA Rules not processed for user `d7d44386436c4dbfa66aeeb5a9b04a71`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:03.405393 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-67d3185d-7aa8-4d2a-9a8a-c159f8d5f6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:03.477576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:03.487010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-578b9f65-59bb-4433-812b-8ffa5b058328 None None] MFA Rules not processed for user `3b3f8de4a3b54d58806506b827cfa89a`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:03.497964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-578b9f65-59bb-4433-812b-8ffa5b058328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:03.741719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 882/1755] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 542 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:03.744155 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca95689-f0cf-4eed-b459-29647262c0f8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:03.744475 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca95689-f0cf-4eed-b459-29647262c0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:03.744725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca95689-f0cf-4eed-b459-29647262c0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:03.760399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 874/1756] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:03.760399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b9f65-59bb-4433-812b-8ffa5b058328 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:03.761629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-578b9f65-59bb-4433-812b-8ffa5b058328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:03.792317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-bca95689-f0cf-4eed-b459-29647262c0f8 None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:03.793763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bca95689-f0cf-4eed-b459-29647262c0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:04.014402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b9f65-59bb-4433-812b-8ffa5b058328 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:04.031911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:04.033315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:04.033951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:04.034818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] PATH_INFO: `/v3/projects/209166afa9024d94be6ed71b48cfac70` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:04.038120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=209166afa9024d94be6ed71b48cfac70)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:04.040654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:04.086355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:04.189403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 883/1757] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:18:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:04.190968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca95689-f0cf-4eed-b459-29647262c0f8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:04.198744 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bca95689-f0cf-4eed-b459-29647262c0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:04.205131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209166afa9024d94be6ed71b48cfac70', 'request_id': 'req-1782533f-ec08-468e-a3ae-0258f77d1fc1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:04.235336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209166afa9024d94be6ed71b48cfac70', 'request_id': 'req-1782533f-ec08-468e-a3ae-0258f77d1fc1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:04.244173 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca95689-f0cf-4eed-b459-29647262c0f8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:04.262751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209166afa9024d94be6ed71b48cfac70', 'request_id': 'req-1782533f-ec08-468e-a3ae-0258f77d1fc1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:04.325148 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] The token cache is being invalidate because project 209166afa9024d94be6ed71b48cfac70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:04.325953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:04.327263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:04.331397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 875/1758] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:03 2022] DELETE /identity/v3/projects/209166afa9024d94be6ed71b48cfac70 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:04.335924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:04.339775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:04.499084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782533f-ec08-468e-a3ae-0258f77d1fc1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:04.503203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2987da7-0a90-4204-b5f9-51724e22765f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:04.505462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2987da7-0a90-4204-b5f9-51724e22765f admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:04.507601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:04.507952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2987da7-0a90-4204-b5f9-51724e22765f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:04.509175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2987da7-0a90-4204-b5f9-51724e22765f admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:04.509348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:04.509922 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:04.510420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] PATH_INFO: `/v3/users/a7e111a4901d466f8c9b856973c7e199` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:04.513615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7e111a4901d466f8c9b856973c7e199)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:04.514151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2987da7-0a90-4204-b5f9-51724e22765f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:04.516005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:04.518532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2987da7-0a90-4204-b5f9-51724e22765f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:04.643567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 884/1759] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:04 2022] POST /identity/v3/projects => generated 369 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:04.648163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2987da7-0a90-4204-b5f9-51724e22765f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:04.651341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7e111a4901d466f8c9b856973c7e199'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:04.654238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c2987da7-0a90-4204-b5f9-51724e22765f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:04.692361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7e111a4901d466f8c9b856973c7e199'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:04.722503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7e111a4901d466f8c9b856973c7e199'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:04.724353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7e111a4901d466f8c9b856973c7e199'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:04.756260 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7e111a4901d466f8c9b856973c7e199'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:04.772943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 876/1760] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:04 2022] DELETE /identity/v3/users/a7e111a4901d466f8c9b856973c7e199 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:04.774505 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:04.776831 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:04.828203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:04.906731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:04.919839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2987da7-0a90-4204-b5f9-51724e22765f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:05.044011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020280f-554f-4dea-b500-263caa798d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:05.044815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020280f-554f-4dea-b500-263caa798d47 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:05.045170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020280f-554f-4dea-b500-263caa798d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:05.045793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020280f-554f-4dea-b500-263caa798d47 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:05.046571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020280f-554f-4dea-b500-263caa798d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:05.050905 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff47759-1bbb-4393-b736-00775777f3bb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:05.055062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020280f-554f-4dea-b500-263caa798d47 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:05.067715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0c793-dbb1-4953-a24b-ebfb024e2998 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:05.068420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0c793-dbb1-4953-a24b-ebfb024e2998 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:05.078457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0c793-dbb1-4953-a24b-ebfb024e2998 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:05.078817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0c793-dbb1-4953-a24b-ebfb024e2998 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:05.086802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dce0c793-dbb1-4953-a24b-ebfb024e2998 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:05.105013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 885/1761] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:04 2022] POST /identity/v3/users => generated 291 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:05.106318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020280f-554f-4dea-b500-263caa798d47 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:05.109810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b020280f-554f-4dea-b500-263caa798d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:05.118341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0c793-dbb1-4953-a24b-ebfb024e2998 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:05.120085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0c793-dbb1-4953-a24b-ebfb024e2998 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:05.146467 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020280f-554f-4dea-b500-263caa798d47 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:05.166997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 877/1762] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:05.178644 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4196b5a-690f-479b-976a-de0636376ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:05.186015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4196b5a-690f-479b-976a-de0636376ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:05.186331 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4196b5a-690f-479b-976a-de0636376ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:05.186616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4196b5a-690f-479b-976a-de0636376ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:05.187571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4196b5a-690f-479b-976a-de0636376ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:05.199753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4196b5a-690f-479b-976a-de0636376ac8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:05.264950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 886/1763] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:05.269983 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0c793-dbb1-4953-a24b-ebfb024e2998 service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:05.287911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0c793-dbb1-4953-a24b-ebfb024e2998 service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:05.303409 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241ef44-05da-4ac6-bffa-a2c8f67a3049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:05.306345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241ef44-05da-4ac6-bffa-a2c8f67a3049 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:05.307888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241ef44-05da-4ac6-bffa-a2c8f67a3049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:05.309421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241ef44-05da-4ac6-bffa-a2c8f67a3049 admin admin] PATH_INFO: `/v3/projects/457ad2bab3764ef8a2bd92949e393b97/users/2c8c1c052f3648e0b11b732974b170e3/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:05.363240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c241ef44-05da-4ac6-bffa-a2c8f67a3049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=457ad2bab3764ef8a2bd92949e393b97, user_id=2c8c1c052f3648e0b11b732974b170e3, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:05.370222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c241ef44-05da-4ac6-bffa-a2c8f67a3049 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:05.425970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4196b5a-690f-479b-976a-de0636376ac8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:05.426973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 878/1764] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:05 2022] PUT /identity/v3/projects/457ad2bab3764ef8a2bd92949e393b97/users/2c8c1c052f3648e0b11b732974b170e3/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:05.436875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4196b5a-690f-479b-976a-de0636376ac8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:05.616375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5fbe6-f8a9-4314-aa79-f7a70c0dc729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:05.617239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5fbe6-f8a9-4314-aa79-f7a70c0dc729 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:05.617900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5fbe6-f8a9-4314-aa79-f7a70c0dc729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:05.618315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5fbe6-f8a9-4314-aa79-f7a70c0dc729 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:05.619204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5fbe6-f8a9-4314-aa79-f7a70c0dc729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:05.621347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5fbe6-f8a9-4314-aa79-f7a70c0dc729 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:05.652931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 887/1765] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:05.657273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241ef44-05da-4ac6-bffa-a2c8f67a3049 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:05.697712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241ef44-05da-4ac6-bffa-a2c8f67a3049 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:05.711966 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11465c0-f087-4f2f-b6e9-85aecda7e389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:05.713136 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11465c0-f087-4f2f-b6e9-85aecda7e389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:05.713678 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11465c0-f087-4f2f-b6e9-85aecda7e389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:05.714173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11465c0-f087-4f2f-b6e9-85aecda7e389 admin admin] PATH_INFO: `/v3/projects/457ad2bab3764ef8a2bd92949e393b97/users/2c8c1c052f3648e0b11b732974b170e3/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:05.716902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11465c0-f087-4f2f-b6e9-85aecda7e389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=457ad2bab3764ef8a2bd92949e393b97, user_id=2c8c1c052f3648e0b11b732974b170e3, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:05.721900 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11465c0-f087-4f2f-b6e9-85aecda7e389 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:05.734940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 879/1766] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:05 2022] PUT /identity/v3/projects/457ad2bab3764ef8a2bd92949e393b97/users/2c8c1c052f3648e0b11b732974b170e3/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:05.746791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe007d-88d6-4205-a80a-990daf5a2dd4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:05.747121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe007d-88d6-4205-a80a-990daf5a2dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:05.747355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe007d-88d6-4205-a80a-990daf5a2dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:05.826881 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-46fe007d-88d6-4205-a80a-990daf5a2dd4 None None] MFA Rules not processed for user `2c8c1c052f3648e0b11b732974b170e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:05.833602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-46fe007d-88d6-4205-a80a-990daf5a2dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:06.019029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:06.082701 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 888/1767] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:05 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:06.095157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11465c0-f087-4f2f-b6e9-85aecda7e389 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:06.098013 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c11465c0-f087-4f2f-b6e9-85aecda7e389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:06.256292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11465c0-f087-4f2f-b6e9-85aecda7e389 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:06.265006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4bfbd-ef4f-4dd2-811b-449047777e20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:06.267106 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4bfbd-ef4f-4dd2-811b-449047777e20 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:06.269743 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4bfbd-ef4f-4dd2-811b-449047777e20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:06.270742 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4bfbd-ef4f-4dd2-811b-449047777e20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:06.304520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4bfbd-ef4f-4dd2-811b-449047777e20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:06.306348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4bfbd-ef4f-4dd2-811b-449047777e20 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:06.339818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 880/1768] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:06 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:06.528007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe007d-88d6-4205-a80a-990daf5a2dd4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:06.555935 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe007d-88d6-4205-a80a-990daf5a2dd4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:06.570158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f975b-fe1d-413f-a4a6-eb747ade9cfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:06.571295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f975b-fe1d-413f-a4a6-eb747ade9cfa service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:06.573094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f975b-fe1d-413f-a4a6-eb747ade9cfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:06.573491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f975b-fe1d-413f-a4a6-eb747ade9cfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:06.586031 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5c8f975b-fe1d-413f-a4a6-eb747ade9cfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:06.674646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4bfbd-ef4f-4dd2-811b-449047777e20 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:06.676323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8ba4bfbd-ef4f-4dd2-811b-449047777e20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:06.835813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f975b-fe1d-413f-a4a6-eb747ade9cfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:06.837815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f975b-fe1d-413f-a4a6-eb747ade9cfa service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:06.845025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4bfbd-ef4f-4dd2-811b-449047777e20 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:06.849756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 889/1769] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:06.850811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f975b-fe1d-413f-a4a6-eb747ade9cfa service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:06.852182 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5c8f975b-fe1d-413f-a4a6-eb747ade9cfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:06.853504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be7db1-3078-47d1-9024-8fa5935d5c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:06.854647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be7db1-3078-47d1-9024-8fa5935d5c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:06.855119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be7db1-3078-47d1-9024-8fa5935d5c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:06.855558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be7db1-3078-47d1-9024-8fa5935d5c2b admin admin] PATH_INFO: `/v3/projects/04c4cb877f884aceac62eb9234d8d262` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:06.857874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be7db1-3078-47d1-9024-8fa5935d5c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=04c4cb877f884aceac62eb9234d8d262)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:06.862792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be7db1-3078-47d1-9024-8fa5935d5c2b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:06.872935 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f975b-fe1d-413f-a4a6-eb747ade9cfa service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:06.884969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12bc37-2867-4845-9e0d-668157df5f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:06.885884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12bc37-2867-4845-9e0d-668157df5f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:06.886172 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12bc37-2867-4845-9e0d-668157df5f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:06.886439 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12bc37-2867-4845-9e0d-668157df5f8c admin admin] PATH_INFO: `/v3/projects/2abf5c445d9347e89431ac7eb3ed8ca1` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:06.898239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d12bc37-2867-4845-9e0d-668157df5f8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2abf5c445d9347e89431ac7eb3ed8ca1)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:06.900588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d12bc37-2867-4845-9e0d-668157df5f8c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:06.903943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:06.969198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f4be7db1-3078-47d1-9024-8fa5935d5c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04c4cb877f884aceac62eb9234d8d262', 'request_id': 'req-f4be7db1-3078-47d1-9024-8fa5935d5c2b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:06.982565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f4be7db1-3078-47d1-9024-8fa5935d5c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04c4cb877f884aceac62eb9234d8d262', 'request_id': 'req-f4be7db1-3078-47d1-9024-8fa5935d5c2b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:06.993312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f4be7db1-3078-47d1-9024-8fa5935d5c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04c4cb877f884aceac62eb9234d8d262', 'request_id': 'req-f4be7db1-3078-47d1-9024-8fa5935d5c2b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:07.005678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1d12bc37-2867-4845-9e0d-668157df5f8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2abf5c445d9347e89431ac7eb3ed8ca1', 'request_id': 'req-1d12bc37-2867-4845-9e0d-668157df5f8c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:07.012312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f4be7db1-3078-47d1-9024-8fa5935d5c2b admin admin] The token cache is being invalidate because project 04c4cb877f884aceac62eb9234d8d262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:07.012548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f4be7db1-3078-47d1-9024-8fa5935d5c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:07.013636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f4be7db1-3078-47d1-9024-8fa5935d5c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:07.015386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 881/1770] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:06 2022] DELETE /identity/v3/projects/04c4cb877f884aceac62eb9234d8d262 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:07.021104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc41eb-d021-4f98-b204-4608ec415393 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.021283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1d12bc37-2867-4845-9e0d-668157df5f8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2abf5c445d9347e89431ac7eb3ed8ca1', 'request_id': 'req-1d12bc37-2867-4845-9e0d-668157df5f8c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:07.021642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc41eb-d021-4f98-b204-4608ec415393 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.022084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc41eb-d021-4f98-b204-4608ec415393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.032812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1d12bc37-2867-4845-9e0d-668157df5f8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2abf5c445d9347e89431ac7eb3ed8ca1', 'request_id': 'req-1d12bc37-2867-4845-9e0d-668157df5f8c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:07.052470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1d12bc37-2867-4845-9e0d-668157df5f8c admin admin] The token cache is being invalidate because project 2abf5c445d9347e89431ac7eb3ed8ca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:07.052786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1d12bc37-2867-4845-9e0d-668157df5f8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:07.053172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ddfc41eb-d021-4f98-b204-4608ec415393 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:07.053849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1d12bc37-2867-4845-9e0d-668157df5f8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:07.054746 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ddfc41eb-d021-4f98-b204-4608ec415393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:07.058100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 890/1771] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:06 2022] DELETE /identity/v3/projects/2abf5c445d9347e89431ac7eb3ed8ca1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:07.060997 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0abf6-a1eb-4f8c-8916-53b4fbae6d58 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.061456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0abf6-a1eb-4f8c-8916-53b4fbae6d58 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.062087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0abf6-a1eb-4f8c-8916-53b4fbae6d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.089060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-04a0abf6-a1eb-4f8c-8916-53b4fbae6d58 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:07.090460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-04a0abf6-a1eb-4f8c-8916-53b4fbae6d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:07.170480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 882/1772] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:07.178565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af9db9-be83-4555-a9b3-b68c72e60b47 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.178565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af9db9-be83-4555-a9b3-b68c72e60b47 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.180674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af9db9-be83-4555-a9b3-b68c72e60b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.209544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-14af9db9-be83-4555-a9b3-b68c72e60b47 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:07.211703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-14af9db9-be83-4555-a9b3-b68c72e60b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:07.225117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:07.230015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 883/1773] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:07.230843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af9db9-be83-4555-a9b3-b68c72e60b47 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:07.237579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 891/1774] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:07.242494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af9db9-be83-4555-a9b3-b68c72e60b47 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:07.245686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737cefe-8abf-4bf1-a92c-72fbcb87b635 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.246968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737cefe-8abf-4bf1-a92c-72fbcb87b635 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.251672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737cefe-8abf-4bf1-a92c-72fbcb87b635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.253460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8da8d-9333-41b2-b24e-fcecdde41a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:07.254355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8da8d-9333-41b2-b24e-fcecdde41a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.254778 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8da8d-9333-41b2-b24e-fcecdde41a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.255173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8da8d-9333-41b2-b24e-fcecdde41a59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.256142 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8da8d-9333-41b2-b24e-fcecdde41a59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:07.256559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8da8d-9333-41b2-b24e-fcecdde41a59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:07.258887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8da8d-9333-41b2-b24e-fcecdde41a59 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:07.293512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 884/1775] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:07.298454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8da8d-9333-41b2-b24e-fcecdde41a59 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:07.301638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5737cefe-8abf-4bf1-a92c-72fbcb87b635 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:07.303500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5737cefe-8abf-4bf1-a92c-72fbcb87b635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:07.315502 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8da8d-9333-41b2-b24e-fcecdde41a59 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:07.318145 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 892/1776] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:07.321011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737cefe-8abf-4bf1-a92c-72fbcb87b635 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:07.333980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737cefe-8abf-4bf1-a92c-72fbcb87b635 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:07.346258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28821020-1041-48f6-a4d3-56c44c48b546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:07.357343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28821020-1041-48f6-a4d3-56c44c48b546 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.357593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28821020-1041-48f6-a4d3-56c44c48b546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.357911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28821020-1041-48f6-a4d3-56c44c48b546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.358626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28821020-1041-48f6-a4d3-56c44c48b546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:07.365831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60da4cd-f0df-4adc-abcb-e83c35daa621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:07.367549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60da4cd-f0df-4adc-abcb-e83c35daa621 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.368064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60da4cd-f0df-4adc-abcb-e83c35daa621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.368568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60da4cd-f0df-4adc-abcb-e83c35daa621 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.369515 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60da4cd-f0df-4adc-abcb-e83c35daa621 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:07.370746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60da4cd-f0df-4adc-abcb-e83c35daa621 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:07.375561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60da4cd-f0df-4adc-abcb-e83c35daa621 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:07.378753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28821020-1041-48f6-a4d3-56c44c48b546 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:07.409927 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 893/1777] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:07.411112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60da4cd-f0df-4adc-abcb-e83c35daa621 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:07.426566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60da4cd-f0df-4adc-abcb-e83c35daa621 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:07.433626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13523343-5121-450a-898f-2c32ebc92862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:07.434504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13523343-5121-450a-898f-2c32ebc92862 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.434795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13523343-5121-450a-898f-2c32ebc92862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.435071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13523343-5121-450a-898f-2c32ebc92862 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.435803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13523343-5121-450a-898f-2c32ebc92862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:07.438158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13523343-5121-450a-898f-2c32ebc92862 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:07.443782 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 885/1778] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:07 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:07.448020 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28821020-1041-48f6-a4d3-56c44c48b546 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:07.511999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 894/1779] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:07 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:07.516023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13523343-5121-450a-898f-2c32ebc92862 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:07.556892 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28821020-1041-48f6-a4d3-56c44c48b546 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:07.570709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:07.654256 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657fb97-a4b3-46bb-89ef-bf84a20fe236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:07.655119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657fb97-a4b3-46bb-89ef-bf84a20fe236 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.655503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657fb97-a4b3-46bb-89ef-bf84a20fe236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.656894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657fb97-a4b3-46bb-89ef-bf84a20fe236 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.660060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657fb97-a4b3-46bb-89ef-bf84a20fe236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:07.662309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657fb97-a4b3-46bb-89ef-bf84a20fe236 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:07.669534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13523343-5121-450a-898f-2c32ebc92862 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:07.677122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9ea2b-5703-41b0-ae6f-521995698942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:07.678008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9ea2b-5703-41b0-ae6f-521995698942 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.678324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9ea2b-5703-41b0-ae6f-521995698942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.678998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9ea2b-5703-41b0-ae6f-521995698942 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.679768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9ea2b-5703-41b0-ae6f-521995698942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:07.683226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9ea2b-5703-41b0-ae6f-521995698942 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:07.707923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 895/1780] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:07 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:07.711816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9ea2b-5703-41b0-ae6f-521995698942 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:07.723402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9ea2b-5703-41b0-ae6f-521995698942 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:07.730364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 886/1781] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:07 2022] POST /identity/v3/users => generated 294 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:07.739821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657fb97-a4b3-46bb-89ef-bf84a20fe236 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:07.740404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d9abf-5a4d-4fb8-9694-814107b1f976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:07.741897 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d9abf-5a4d-4fb8-9694-814107b1f976 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.742306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d9abf-5a4d-4fb8-9694-814107b1f976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.742729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d9abf-5a4d-4fb8-9694-814107b1f976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.744612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d9abf-5a4d-4fb8-9694-814107b1f976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:07.748362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d9abf-5a4d-4fb8-9694-814107b1f976 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:07.766132 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657fb97-a4b3-46bb-89ef-bf84a20fe236 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:07.767397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 896/1782] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:07.771430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d9abf-5a4d-4fb8-9694-814107b1f976 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:07.785098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d9abf-5a4d-4fb8-9694-814107b1f976 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:07.786766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93427b4c-c1c4-40ba-8518-5942338409e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:07.789332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93427b4c-c1c4-40ba-8518-5942338409e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.789775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93427b4c-c1c4-40ba-8518-5942338409e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.790141 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93427b4c-c1c4-40ba-8518-5942338409e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.792124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43330135-243c-44e2-822f-26e510a730aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:07.792263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93427b4c-c1c4-40ba-8518-5942338409e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:07.793078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43330135-243c-44e2-822f-26e510a730aa admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.793673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43330135-243c-44e2-822f-26e510a730aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.794403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43330135-243c-44e2-822f-26e510a730aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.795834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93427b4c-c1c4-40ba-8518-5942338409e5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:07.796079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43330135-243c-44e2-822f-26e510a730aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:07.797683 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43330135-243c-44e2-822f-26e510a730aa admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:07.816023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 897/1783] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:07.820370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43330135-243c-44e2-822f-26e510a730aa admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:07.827250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 887/1784] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:07.832896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93427b4c-c1c4-40ba-8518-5942338409e5 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:07.836687 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43330135-243c-44e2-822f-26e510a730aa admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:07.842392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e0b59-75a6-460c-a6c3-71b0b3b4b7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:07.843535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e0b59-75a6-460c-a6c3-71b0b3b4b7b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.843937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e0b59-75a6-460c-a6c3-71b0b3b4b7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.844502 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e0b59-75a6-460c-a6c3-71b0b3b4b7b3 admin admin] PATH_INFO: `/v3/projects/b906dcf1af434eb79ee544471ee21382/users/96ff01b0d0d64d05af2d8fbefa129bfa/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.852405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93427b4c-c1c4-40ba-8518-5942338409e5 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:07.857457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ec896-acbb-4334-a1f4-a5eb6f8bad28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:07.858688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ec896-acbb-4334-a1f4-a5eb6f8bad28 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.858973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ec896-acbb-4334-a1f4-a5eb6f8bad28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.859250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ec896-acbb-4334-a1f4-a5eb6f8bad28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.859968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ec896-acbb-4334-a1f4-a5eb6f8bad28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:07.862310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ec896-acbb-4334-a1f4-a5eb6f8bad28 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:07.882464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e0b59-75a6-460c-a6c3-71b0b3b4b7b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b906dcf1af434eb79ee544471ee21382, user_id=96ff01b0d0d64d05af2d8fbefa129bfa, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:07.886010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e0b59-75a6-460c-a6c3-71b0b3b4b7b3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:07.886426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 888/1785] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:07.897405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ec896-acbb-4334-a1f4-a5eb6f8bad28 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:07.921597 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ec896-acbb-4334-a1f4-a5eb6f8bad28 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:07.922360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 898/1786] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:07 2022] PUT /identity/v3/projects/b906dcf1af434eb79ee544471ee21382/users/96ff01b0d0d64d05af2d8fbefa129bfa/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:07.933426 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39ff6d-636b-4fbc-8a2f-6f04dcc08fe6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:07.933942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39ff6d-636b-4fbc-8a2f-6f04dcc08fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:07.934225 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39ff6d-636b-4fbc-8a2f-6f04dcc08fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:07.985464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ac39ff6d-636b-4fbc-8a2f-6f04dcc08fe6 None None] MFA Rules not processed for user `96ff01b0d0d64d05af2d8fbefa129bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:07.987603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ac39ff6d-636b-4fbc-8a2f-6f04dcc08fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:08.021255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe3223-7daa-4c2f-981b-045216592554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.023040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe3223-7daa-4c2f-981b-045216592554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.023341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe3223-7daa-4c2f-981b-045216592554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.023615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe3223-7daa-4c2f-981b-045216592554 admin admin] PATH_INFO: `/v3/projects/4820bf2a74694a2c895bcab9054ae7d3/users/6c1bec8780214221af76feb209cf84dc/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.045237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe3223-7daa-4c2f-981b-045216592554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4820bf2a74694a2c895bcab9054ae7d3, user_id=6c1bec8780214221af76feb209cf84dc, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.049277 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe3223-7daa-4c2f-981b-045216592554 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.056213 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:08.073953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 889/1787] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:07 2022] PUT /identity/v3/projects/4820bf2a74694a2c895bcab9054ae7d3/users/6c1bec8780214221af76feb209cf84dc/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:08.084052 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 899/1788] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:08.085410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ae421-3348-4335-8c45-8f1108b8b11d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.085705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ae421-3348-4335-8c45-8f1108b8b11d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.086066 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ae421-3348-4335-8c45-8f1108b8b11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.088447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39ff6d-636b-4fbc-8a2f-6f04dcc08fe6 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.100093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39ff6d-636b-4fbc-8a2f-6f04dcc08fe6 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.116871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b48ae421-3348-4335-8c45-8f1108b8b11d None None] MFA Rules not processed for user `6c1bec8780214221af76feb209cf84dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:08.118129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b48ae421-3348-4335-8c45-8f1108b8b11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:08.186932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:08.192321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe74b5-b061-492b-b920-92c7edaf251e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.198135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe74b5-b061-492b-b920-92c7edaf251e admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.199969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe74b5-b061-492b-b920-92c7edaf251e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.200680 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe74b5-b061-492b-b920-92c7edaf251e admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.203315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe74b5-b061-492b-b920-92c7edaf251e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.209581 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe74b5-b061-492b-b920-92c7edaf251e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.219432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 890/1789] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:08.223591 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ae421-3348-4335-8c45-8f1108b8b11d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.237115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ae421-3348-4335-8c45-8f1108b8b11d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.245301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c373cd8-a9a9-4949-b8b2-24d8c28fdc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.246111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c373cd8-a9a9-4949-b8b2-24d8c28fdc3a admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.246546 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c373cd8-a9a9-4949-b8b2-24d8c28fdc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.246830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c373cd8-a9a9-4949-b8b2-24d8c28fdc3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.247642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c373cd8-a9a9-4949-b8b2-24d8c28fdc3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.250100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c373cd8-a9a9-4949-b8b2-24d8c28fdc3a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.257955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 900/1790] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:08 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:08.261852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe74b5-b061-492b-b920-92c7edaf251e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.294937 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 891/1791] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:08 2022] POST /identity/v3/projects => generated 389 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:08.298950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c373cd8-a9a9-4949-b8b2-24d8c28fdc3a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.382990 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe74b5-b061-492b-b920-92c7edaf251e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.398578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c373cd8-a9a9-4949-b8b2-24d8c28fdc3a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.411077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c46d0-0df6-4fdf-b184-6c6e934603a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.411899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c46d0-0df6-4fdf-b184-6c6e934603a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.412329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c46d0-0df6-4fdf-b184-6c6e934603a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.412677 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c46d0-0df6-4fdf-b184-6c6e934603a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.413800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c46d0-0df6-4fdf-b184-6c6e934603a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.418154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c46d0-0df6-4fdf-b184-6c6e934603a2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.443789 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 892/1792] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:08 2022] POST /identity/v3/users => generated 301 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:08.447296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867696b-1f16-4c41-a33b-3f4d9dc85703 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.447921 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867696b-1f16-4c41-a33b-3f4d9dc85703 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.448307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867696b-1f16-4c41-a33b-3f4d9dc85703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.465461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0f99e-3c98-4890-b6e8-f595edaae2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.466183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0f99e-3c98-4890-b6e8-f595edaae2d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.466463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0f99e-3c98-4890-b6e8-f595edaae2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.466741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0f99e-3c98-4890-b6e8-f595edaae2d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.467437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0f99e-3c98-4890-b6e8-f595edaae2d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.469369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0f99e-3c98-4890-b6e8-f595edaae2d8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.469869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5867696b-1f16-4c41-a33b-3f4d9dc85703 None None] MFA Rules not processed for user `a20f6932d763461ea26c2d60ed58a59f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:08.471895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5867696b-1f16-4c41-a33b-3f4d9dc85703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:08.494201 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 901/1793] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:08 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:08.495109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0f99e-3c98-4890-b6e8-f595edaae2d8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.509240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0f99e-3c98-4890-b6e8-f595edaae2d8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.515451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc7a74-1887-4185-9e97-56a9037b6c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.516321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc7a74-1887-4185-9e97-56a9037b6c08 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.516690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc7a74-1887-4185-9e97-56a9037b6c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.517166 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc7a74-1887-4185-9e97-56a9037b6c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.518503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc7a74-1887-4185-9e97-56a9037b6c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.520184 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc7a74-1887-4185-9e97-56a9037b6c08 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.532677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 902/1794] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:08.535075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc7a74-1887-4185-9e97-56a9037b6c08 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.541258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:08.546964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc7a74-1887-4185-9e97-56a9037b6c08 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.572795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 893/1795] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:08.573814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867696b-1f16-4c41-a33b-3f4d9dc85703 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.574791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9654c3-9761-4292-b8c9-a46ca4ffc406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.575465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9654c3-9761-4292-b8c9-a46ca4ffc406 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.575740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9654c3-9761-4292-b8c9-a46ca4ffc406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.576007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9654c3-9761-4292-b8c9-a46ca4ffc406 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.576761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9654c3-9761-4292-b8c9-a46ca4ffc406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.578448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9654c3-9761-4292-b8c9-a46ca4ffc406 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.609657 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 903/1796] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:08.618859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867696b-1f16-4c41-a33b-3f4d9dc85703 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.619216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9654c3-9761-4292-b8c9-a46ca4ffc406 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.628111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8793b0f-9fa3-417e-a572-5d5c1c72d471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.629521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793b0f-9fa3-417e-a572-5d5c1c72d471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.629686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793b0f-9fa3-417e-a572-5d5c1c72d471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.630069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793b0f-9fa3-417e-a572-5d5c1c72d471 admin admin] PATH_INFO: `/v3/projects/1d87fd8f2ce248f8a2ab37b96e6192d7/users/67f08cd762524e54badda2b4b88e7b01/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.648501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9654c3-9761-4292-b8c9-a46ca4ffc406 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.649341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8793b0f-9fa3-417e-a572-5d5c1c72d471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d87fd8f2ce248f8a2ab37b96e6192d7, user_id=67f08cd762524e54badda2b4b88e7b01, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.652287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8793b0f-9fa3-417e-a572-5d5c1c72d471 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.659960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfe8fe-b385-4f3e-a19e-821a92ec94f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.660650 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfe8fe-b385-4f3e-a19e-821a92ec94f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.660970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfe8fe-b385-4f3e-a19e-821a92ec94f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.661248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfe8fe-b385-4f3e-a19e-821a92ec94f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.662021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfe8fe-b385-4f3e-a19e-821a92ec94f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.663644 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfe8fe-b385-4f3e-a19e-821a92ec94f9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.685567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 894/1797] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:08 2022] PUT /identity/v3/projects/1d87fd8f2ce248f8a2ab37b96e6192d7/users/67f08cd762524e54badda2b4b88e7b01/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:08.692487 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8793b0f-9fa3-417e-a572-5d5c1c72d471 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.693648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 904/1798] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:08.696156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfe8fe-b385-4f3e-a19e-821a92ec94f9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.712682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfe8fe-b385-4f3e-a19e-821a92ec94f9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.757108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:08.781557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8793b0f-9fa3-417e-a572-5d5c1c72d471 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.785326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3941251-07b7-483a-b767-c8bff263d7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.786255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3941251-07b7-483a-b767-c8bff263d7b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.786534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3941251-07b7-483a-b767-c8bff263d7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.786809 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3941251-07b7-483a-b767-c8bff263d7b3 admin admin] PATH_INFO: `/v3/projects/c890831defa24509a192fd5aa13fc6a8/users/3f9f303710b848178db0902cd23e9601/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.787128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fd0cb-0157-4f5f-9b08-1ffc730e1604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.787808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fd0cb-0157-4f5f-9b08-1ffc730e1604 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.788092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fd0cb-0157-4f5f-9b08-1ffc730e1604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.788365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fd0cb-0157-4f5f-9b08-1ffc730e1604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.789323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fd0cb-0157-4f5f-9b08-1ffc730e1604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.791752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fd0cb-0157-4f5f-9b08-1ffc730e1604 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.804208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 895/1799] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:08.807817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3941251-07b7-483a-b767-c8bff263d7b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c890831defa24509a192fd5aa13fc6a8, user_id=3f9f303710b848178db0902cd23e9601, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.810582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3941251-07b7-483a-b767-c8bff263d7b3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.810803 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fd0cb-0157-4f5f-9b08-1ffc730e1604 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.824617 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fd0cb-0157-4f5f-9b08-1ffc730e1604 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.828314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 905/1800] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:08 2022] PUT /identity/v3/projects/c890831defa24509a192fd5aa13fc6a8/users/3f9f303710b848178db0902cd23e9601/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:08.840279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c75bd6-5f9d-4932-a8e7-f5d98e8fc505 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.840565 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c75bd6-5f9d-4932-a8e7-f5d98e8fc505 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.840847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c75bd6-5f9d-4932-a8e7-f5d98e8fc505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.868813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a5c75bd6-5f9d-4932-a8e7-f5d98e8fc505 None None] MFA Rules not processed for user `3f9f303710b848178db0902cd23e9601`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:08.870104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a5c75bd6-5f9d-4932-a8e7-f5d98e8fc505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:08.895469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bea16-7cec-491f-9a73-2e0dcd804d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.897574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bea16-7cec-491f-9a73-2e0dcd804d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.900761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bea16-7cec-491f-9a73-2e0dcd804d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.901069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bea16-7cec-491f-9a73-2e0dcd804d53 admin admin] PATH_INFO: `/v3/projects/1d87fd8f2ce248f8a2ab37b96e6192d7/users/67f08cd762524e54badda2b4b88e7b01/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.914695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15bea16-7cec-491f-9a73-2e0dcd804d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d87fd8f2ce248f8a2ab37b96e6192d7, user_id=67f08cd762524e54badda2b4b88e7b01, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.917366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15bea16-7cec-491f-9a73-2e0dcd804d53 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.932291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 896/1801] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:08 2022] PUT /identity/v3/projects/1d87fd8f2ce248f8a2ab37b96e6192d7/users/67f08cd762524e54badda2b4b88e7b01/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:08.944194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a447b1-ff39-4701-985d-2377c252cd1d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.944482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a447b1-ff39-4701-985d-2377c252cd1d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.944754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a447b1-ff39-4701-985d-2377c252cd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.957079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 906/1802] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:08.961533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c75bd6-5f9d-4932-a8e7-f5d98e8fc505 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.973051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c75bd6-5f9d-4932-a8e7-f5d98e8fc505 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.974440 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-57a447b1-ff39-4701-985d-2377c252cd1d None None] MFA Rules not processed for user `67f08cd762524e54badda2b4b88e7b01`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:08.975694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-57a447b1-ff39-4701-985d-2377c252cd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:09.025969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:09.037324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f53c6-225e-49f5-a6a1-9686d2dadcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:09.038254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f53c6-225e-49f5-a6a1-9686d2dadcb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:09.038578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f53c6-225e-49f5-a6a1-9686d2dadcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:09.038953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f53c6-225e-49f5-a6a1-9686d2dadcb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:09.039744 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f53c6-225e-49f5-a6a1-9686d2dadcb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:09.041818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f53c6-225e-49f5-a6a1-9686d2dadcb7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:09.061457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 897/1803] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:09.069932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a447b1-ff39-4701-985d-2377c252cd1d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:09.071741 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-57a447b1-ff39-4701-985d-2377c252cd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:09.087438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 907/1804] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:08 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:09.088450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f53c6-225e-49f5-a6a1-9686d2dadcb7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:09.137526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a447b1-ff39-4701-985d-2377c252cd1d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:09.147860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:09.166677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f53c6-225e-49f5-a6a1-9686d2dadcb7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:09.172026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af871fe3-e7f5-422d-af9a-295c661050b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:09.172813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af871fe3-e7f5-422d-af9a-295c661050b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:09.173418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af871fe3-e7f5-422d-af9a-295c661050b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:09.173862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af871fe3-e7f5-422d-af9a-295c661050b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:09.174838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af871fe3-e7f5-422d-af9a-295c661050b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:09.176725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af871fe3-e7f5-422d-af9a-295c661050b9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:09.197703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76b802-2d3d-41bf-9681-2f38e607e639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:09.198493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76b802-2d3d-41bf-9681-2f38e607e639 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:09.198778 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76b802-2d3d-41bf-9681-2f38e607e639 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:09.199051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76b802-2d3d-41bf-9681-2f38e607e639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:09.201480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 908/1805] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:09 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:09.206779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af871fe3-e7f5-422d-af9a-295c661050b9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:09.224133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af871fe3-e7f5-422d-af9a-295c661050b9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:09.231140 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b1472-7330-4a97-bd26-b594ab76b3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:09.232118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b1472-7330-4a97-bd26-b594ab76b3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:09.232569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b1472-7330-4a97-bd26-b594ab76b3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:09.233069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b1472-7330-4a97-bd26-b594ab76b3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:09.234117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b1472-7330-4a97-bd26-b594ab76b3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:09.237171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b1472-7330-4a97-bd26-b594ab76b3b5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:09.253097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 909/1806] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:09.259189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b1472-7330-4a97-bd26-b594ab76b3b5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:09.276623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b1472-7330-4a97-bd26-b594ab76b3b5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:09.285164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061d941-a78c-4b09-bbb2-1b0800ebc63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:09.286154 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061d941-a78c-4b09-bbb2-1b0800ebc63c admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:09.286449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061d941-a78c-4b09-bbb2-1b0800ebc63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:09.286722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061d941-a78c-4b09-bbb2-1b0800ebc63c admin admin] PATH_INFO: `/v3/projects/a71c8377119c4b068d68bc138ef15c23/users/1a2f2fd5758445eaa02ef0d13ef27240/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:09.314907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061d941-a78c-4b09-bbb2-1b0800ebc63c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71c8377119c4b068d68bc138ef15c23, user_id=1a2f2fd5758445eaa02ef0d13ef27240, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:09.321040 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061d941-a78c-4b09-bbb2-1b0800ebc63c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:09.334572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76b802-2d3d-41bf-9681-2f38e607e639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:09.337004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76b802-2d3d-41bf-9681-2f38e607e639 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:09.359801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 898/1807] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:18:09.363112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 910/1808] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:09 2022] PUT /identity/v3/projects/a71c8377119c4b068d68bc138ef15c23/users/1a2f2fd5758445eaa02ef0d13ef27240/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:09.366841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76b802-2d3d-41bf-9681-2f38e607e639 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:09.402606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76b802-2d3d-41bf-9681-2f38e607e639 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:09.508046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf28d18-9e5f-4f7d-b354-137ddb126991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:09.509272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf28d18-9e5f-4f7d-b354-137ddb126991 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:09.509699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf28d18-9e5f-4f7d-b354-137ddb126991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:09.510388 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf28d18-9e5f-4f7d-b354-137ddb126991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:09.511249 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf28d18-9e5f-4f7d-b354-137ddb126991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:09.513044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf28d18-9e5f-4f7d-b354-137ddb126991 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:09.539781 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 899/1809] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:09.545891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061d941-a78c-4b09-bbb2-1b0800ebc63c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:09.575362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061d941-a78c-4b09-bbb2-1b0800ebc63c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:09.587125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b40d0-b60c-4d00-8626-b29421350458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:09.588112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b40d0-b60c-4d00-8626-b29421350458 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:09.589407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b40d0-b60c-4d00-8626-b29421350458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:09.590049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b40d0-b60c-4d00-8626-b29421350458 admin admin] PATH_INFO: `/v3/projects/a71c8377119c4b068d68bc138ef15c23/users/1a2f2fd5758445eaa02ef0d13ef27240/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:09.594655 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b40d0-b60c-4d00-8626-b29421350458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71c8377119c4b068d68bc138ef15c23, user_id=1a2f2fd5758445eaa02ef0d13ef27240, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:09.598012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b40d0-b60c-4d00-8626-b29421350458 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:09.621712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 911/1810] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:09 2022] PUT /identity/v3/projects/a71c8377119c4b068d68bc138ef15c23/users/1a2f2fd5758445eaa02ef0d13ef27240/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:09.623005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bdc1e-8b9c-48bb-a32c-b90ee736a548 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:09.623472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bdc1e-8b9c-48bb-a32c-b90ee736a548 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:09.623869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bdc1e-8b9c-48bb-a32c-b90ee736a548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:09.703025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-481bdc1e-8b9c-48bb-a32c-b90ee736a548 None None] MFA Rules not processed for user `1a2f2fd5758445eaa02ef0d13ef27240`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:09.709427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-481bdc1e-8b9c-48bb-a32c-b90ee736a548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:09.756461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b40d0-b60c-4d00-8626-b29421350458 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:09.872694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:09.955504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 900/1811] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:09.959133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b40d0-b60c-4d00-8626-b29421350458 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:09.972735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bdc1e-8b9c-48bb-a32c-b90ee736a548 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:09.988136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e77e7-8bc8-49a7-8482-71f7c7d4a665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:09.989236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e77e7-8bc8-49a7-8482-71f7c7d4a665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:09.989607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e77e7-8bc8-49a7-8482-71f7c7d4a665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:09.989998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e77e7-8bc8-49a7-8482-71f7c7d4a665 admin admin] PATH_INFO: `/v3/users/6c1bec8780214221af76feb209cf84dc` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:09.995409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e77e7-8bc8-49a7-8482-71f7c7d4a665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c1bec8780214221af76feb209cf84dc)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:10.002819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e77e7-8bc8-49a7-8482-71f7c7d4a665 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:10.101583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:10.157816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-923e77e7-8bc8-49a7-8482-71f7c7d4a665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c1bec8780214221af76feb209cf84dc'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:10.169656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-923e77e7-8bc8-49a7-8482-71f7c7d4a665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c1bec8780214221af76feb209cf84dc'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:10.170904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-923e77e7-8bc8-49a7-8482-71f7c7d4a665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c1bec8780214221af76feb209cf84dc'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:10.172348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-923e77e7-8bc8-49a7-8482-71f7c7d4a665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c1bec8780214221af76feb209cf84dc'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:10.176923 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bdc1e-8b9c-48bb-a32c-b90ee736a548 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:10.189903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2f1e6-e141-4095-9825-c293ba5cd8fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:10.192378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2f1e6-e141-4095-9825-c293ba5cd8fc service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:10.193508 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2f1e6-e141-4095-9825-c293ba5cd8fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:10.194970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2f1e6-e141-4095-9825-c293ba5cd8fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:10.202161 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8ca2f1e6-e141-4095-9825-c293ba5cd8fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:10.221157 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-923e77e7-8bc8-49a7-8482-71f7c7d4a665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c1bec8780214221af76feb209cf84dc'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:10.247596 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2f1e6-e141-4095-9825-c293ba5cd8fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:10.249778 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2f1e6-e141-4095-9825-c293ba5cd8fc service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:10.254992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 912/1812] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:09 2022] DELETE /identity/v3/users/6c1bec8780214221af76feb209cf84dc => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:10.260792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3841622-c8c4-4adc-b90a-b0372193144e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:10.261465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3841622-c8c4-4adc-b90a-b0372193144e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:10.261771 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3841622-c8c4-4adc-b90a-b0372193144e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:10.297363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 901/1813] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:10.323136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b3841622-c8c4-4adc-b90a-b0372193144e None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:10.325978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b3841622-c8c4-4adc-b90a-b0372193144e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:10.535791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 913/1814] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:10.555368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2f1e6-e141-4095-9825-c293ba5cd8fc service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:10.557015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8ca2f1e6-e141-4095-9825-c293ba5cd8fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:10.643814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2f1e6-e141-4095-9825-c293ba5cd8fc service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:10.773287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f454d-a556-4923-9dc5-13894baa8d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:10.774718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f454d-a556-4923-9dc5-13894baa8d74 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:10.775560 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f454d-a556-4923-9dc5-13894baa8d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:10.777922 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f454d-a556-4923-9dc5-13894baa8d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:10.803706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f454d-a556-4923-9dc5-13894baa8d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:10.807780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f454d-a556-4923-9dc5-13894baa8d74 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:10.826780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 902/1815] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:11.283389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3841622-c8c4-4adc-b90a-b0372193144e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:11.285433 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b3841622-c8c4-4adc-b90a-b0372193144e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:11.426899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3841622-c8c4-4adc-b90a-b0372193144e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:11.432248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc77cd9-2acb-4c7a-b876-6200a839bacb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:11.433128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc77cd9-2acb-4c7a-b876-6200a839bacb service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:11.434078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc77cd9-2acb-4c7a-b876-6200a839bacb service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:11.446107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc77cd9-2acb-4c7a-b876-6200a839bacb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:11.451894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8fc77cd9-2acb-4c7a-b876-6200a839bacb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:11.484103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f454d-a556-4923-9dc5-13894baa8d74 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:11.487345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-953f454d-a556-4923-9dc5-13894baa8d74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:11.514153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f454d-a556-4923-9dc5-13894baa8d74 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:11.531212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:11.532087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:11.532367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:11.532656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] PATH_INFO: `/v3/projects/4820bf2a74694a2c895bcab9054ae7d3` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:11.534164 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4820bf2a74694a2c895bcab9054ae7d3)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:11.536114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:11.588305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:11.592208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc77cd9-2acb-4c7a-b876-6200a839bacb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:11.594375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc77cd9-2acb-4c7a-b876-6200a839bacb service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:11.606408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 914/1816] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:11.627874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:11.635876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4820bf2a74694a2c895bcab9054ae7d3', 'request_id': 'req-fd7b03bd-2e91-4315-b061-798d419713fd'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:11.656136 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4820bf2a74694a2c895bcab9054ae7d3', 'request_id': 'req-fd7b03bd-2e91-4315-b061-798d419713fd'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:11.666535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4820bf2a74694a2c895bcab9054ae7d3', 'request_id': 'req-fd7b03bd-2e91-4315-b061-798d419713fd'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:11.680085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] The token cache is being invalidate because project 4820bf2a74694a2c895bcab9054ae7d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:11.680692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:11.681536 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:11.686473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 903/1817] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:11 2022] DELETE /identity/v3/projects/4820bf2a74694a2c895bcab9054ae7d3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:11.691625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc77cd9-2acb-4c7a-b876-6200a839bacb service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:11.707790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc77cd9-2acb-4c7a-b876-6200a839bacb service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:11.774494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c08b9-9c6b-4c25-a085-6319e93dacf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:11.775380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c08b9-9c6b-4c25-a085-6319e93dacf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:11.775666 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c08b9-9c6b-4c25-a085-6319e93dacf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:11.775952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c08b9-9c6b-4c25-a085-6319e93dacf7 admin admin] PATH_INFO: `/v3/users/67f08cd762524e54badda2b4b88e7b01` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:11.777190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c08b9-9c6b-4c25-a085-6319e93dacf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67f08cd762524e54badda2b4b88e7b01)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:11.779418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c08b9-9c6b-4c25-a085-6319e93dacf7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:11.842170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-139c08b9-9c6b-4c25-a085-6319e93dacf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67f08cd762524e54badda2b4b88e7b01'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:11.850484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-139c08b9-9c6b-4c25-a085-6319e93dacf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67f08cd762524e54badda2b4b88e7b01'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:11.851263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-139c08b9-9c6b-4c25-a085-6319e93dacf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67f08cd762524e54badda2b4b88e7b01'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:11.852042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-139c08b9-9c6b-4c25-a085-6319e93dacf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67f08cd762524e54badda2b4b88e7b01'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:11.865875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-139c08b9-9c6b-4c25-a085-6319e93dacf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67f08cd762524e54badda2b4b88e7b01'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:11.879443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 915/1818] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:11 2022] DELETE /identity/v3/users/67f08cd762524e54badda2b4b88e7b01 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:12.187725 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:12.190162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:12.208724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b03bd-2e91-4315-b061-798d419713fd admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:12.267145 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:12.267991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:12.268272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:12.268543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11 admin admin] PATH_INFO: `/v3/projects/1d87fd8f2ce248f8a2ab37b96e6192d7` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:12.269864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d87fd8f2ce248f8a2ab37b96e6192d7)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:12.271907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:12.321560 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d87fd8f2ce248f8a2ab37b96e6192d7', 'request_id': 'req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:12.329062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d87fd8f2ce248f8a2ab37b96e6192d7', 'request_id': 'req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:12.335347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d87fd8f2ce248f8a2ab37b96e6192d7', 'request_id': 'req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:12.347134 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11 admin admin] The token cache is being invalidate because project 1d87fd8f2ce248f8a2ab37b96e6192d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:12.347418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:12.348050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ce2e56cb-9bf5-422b-8528-7c1734c0bf11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:12.349578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 904/1819] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:12 2022] DELETE /identity/v3/projects/1d87fd8f2ce248f8a2ab37b96e6192d7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:12.354648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdb0fb-4414-45e0-af4b-8e03b5920fd6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:12.355104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdb0fb-4414-45e0-af4b-8e03b5920fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:12.355207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdb0fb-4414-45e0-af4b-8e03b5920fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:12.373222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5ccdb0fb-4414-45e0-af4b-8e03b5920fd6 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:12.374517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5ccdb0fb-4414-45e0-af4b-8e03b5920fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:12.433711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 916/1820] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:12.441636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf521af-7a8a-4d58-8330-c11570bbb8ff None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:12.441956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf521af-7a8a-4d58-8330-c11570bbb8ff None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:12.442233 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf521af-7a8a-4d58-8330-c11570bbb8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:12.460468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-dbf521af-7a8a-4d58-8330-c11570bbb8ff None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:12.461664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dbf521af-7a8a-4d58-8330-c11570bbb8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:12.469460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 905/1821] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:12.474443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdb0fb-4414-45e0-af4b-8e03b5920fd6 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:12.488404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdb0fb-4414-45e0-af4b-8e03b5920fd6 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:12.492952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106e620-2cee-4d1f-b63f-38d11b4d9afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:12.493607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106e620-2cee-4d1f-b63f-38d11b4d9afa admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:12.493918 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106e620-2cee-4d1f-b63f-38d11b4d9afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:12.494190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106e620-2cee-4d1f-b63f-38d11b4d9afa admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:12.494910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106e620-2cee-4d1f-b63f-38d11b4d9afa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:12.495211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106e620-2cee-4d1f-b63f-38d11b4d9afa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:12.496826 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106e620-2cee-4d1f-b63f-38d11b4d9afa admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:12.505985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:12.509575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 917/1822] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:12.514127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf521af-7a8a-4d58-8330-c11570bbb8ff None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:12.524252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf521af-7a8a-4d58-8330-c11570bbb8ff None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:12.528841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f913432-cace-43b1-a163-746206ed4465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:12.529492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f913432-cace-43b1-a163-746206ed4465 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:12.529806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f913432-cace-43b1-a163-746206ed4465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:12.530083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f913432-cace-43b1-a163-746206ed4465 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:12.530787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f913432-cace-43b1-a163-746206ed4465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:12.533345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f913432-cace-43b1-a163-746206ed4465 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:12.558928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 906/1823] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:12 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:12.563149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106e620-2cee-4d1f-b63f-38d11b4d9afa admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:12.573356 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106e620-2cee-4d1f-b63f-38d11b4d9afa admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:12.628896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878eb3b-e9c9-44d5-8815-b714ae4a2135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:12.629484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878eb3b-e9c9-44d5-8815-b714ae4a2135 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:12.629793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878eb3b-e9c9-44d5-8815-b714ae4a2135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:12.630078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878eb3b-e9c9-44d5-8815-b714ae4a2135 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:12.630773 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878eb3b-e9c9-44d5-8815-b714ae4a2135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:12.632713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878eb3b-e9c9-44d5-8815-b714ae4a2135 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:12.653567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 918/1824] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:12 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:12.658911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f913432-cace-43b1-a163-746206ed4465 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:12.669289 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f913432-cace-43b1-a163-746206ed4465 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:12.674627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6fdea-ee41-4c57-bf55-f310038f7224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:12.675286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6fdea-ee41-4c57-bf55-f310038f7224 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:12.675563 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6fdea-ee41-4c57-bf55-f310038f7224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:12.675836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6fdea-ee41-4c57-bf55-f310038f7224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:12.676547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6fdea-ee41-4c57-bf55-f310038f7224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:12.678588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6fdea-ee41-4c57-bf55-f310038f7224 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:12.690918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 907/1825] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:12.694719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878eb3b-e9c9-44d5-8815-b714ae4a2135 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:12.705532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878eb3b-e9c9-44d5-8815-b714ae4a2135 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:12.710201 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba7f8a-a0a9-4d9e-96b8-c142ab82ae6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:12.710850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba7f8a-a0a9-4d9e-96b8-c142ab82ae6f admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:12.711127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba7f8a-a0a9-4d9e-96b8-c142ab82ae6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:12.711395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba7f8a-a0a9-4d9e-96b8-c142ab82ae6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:12.712113 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba7f8a-a0a9-4d9e-96b8-c142ab82ae6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:12.713709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba7f8a-a0a9-4d9e-96b8-c142ab82ae6f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:12.725165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 919/1826] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:12.731588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6fdea-ee41-4c57-bf55-f310038f7224 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:12.741869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6fdea-ee41-4c57-bf55-f310038f7224 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:12.747715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d243d-9c55-4afc-bb14-0b1015dcefa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:12.748597 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d243d-9c55-4afc-bb14-0b1015dcefa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:12.748881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d243d-9c55-4afc-bb14-0b1015dcefa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:12.749148 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d243d-9c55-4afc-bb14-0b1015dcefa7 admin admin] PATH_INFO: `/v3/projects/b96225d0b0e94664863b975d748d2c50/users/ce46efbd97b04cb8894baae2e19c0972/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:12.769258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d243d-9c55-4afc-bb14-0b1015dcefa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96225d0b0e94664863b975d748d2c50, user_id=ce46efbd97b04cb8894baae2e19c0972, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:12.771929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d243d-9c55-4afc-bb14-0b1015dcefa7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:12.774418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:12.788779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 908/1827] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:12 2022] PUT /identity/v3/projects/b96225d0b0e94664863b975d748d2c50/users/ce46efbd97b04cb8894baae2e19c0972/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:12.800579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b743b6b-346e-4892-a05b-a64a4ebe068a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:12.800871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b743b6b-346e-4892-a05b-a64a4ebe068a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:12.801153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b743b6b-346e-4892-a05b-a64a4ebe068a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:12.828277 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2b743b6b-346e-4892-a05b-a64a4ebe068a None None] MFA Rules not processed for user `ce46efbd97b04cb8894baae2e19c0972`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:12.830103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2b743b6b-346e-4892-a05b-a64a4ebe068a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:12.900201 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 920/1828] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:12.911984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d243d-9c55-4afc-bb14-0b1015dcefa7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:12.913671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2e6d243d-9c55-4afc-bb14-0b1015dcefa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:12.970005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d243d-9c55-4afc-bb14-0b1015dcefa7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:12.973809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ed4fd-2027-4462-bd88-5415c8284330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:12.974455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ed4fd-2027-4462-bd88-5415c8284330 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:12.974735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ed4fd-2027-4462-bd88-5415c8284330 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:12.975004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ed4fd-2027-4462-bd88-5415c8284330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:12.983494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ed4fd-2027-4462-bd88-5415c8284330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:12.984912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ed4fd-2027-4462-bd88-5415c8284330 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:12.996698 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 909/1829] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:13.016307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b743b6b-346e-4892-a05b-a64a4ebe068a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:13.086792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b743b6b-346e-4892-a05b-a64a4ebe068a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:13.091348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:13.092218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:13.092498 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:13.092772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] PATH_INFO: `/v3/users/ce46efbd97b04cb8894baae2e19c0972` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:13.093991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce46efbd97b04cb8894baae2e19c0972)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:13.095995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:13.106573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:13.142783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce46efbd97b04cb8894baae2e19c0972'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:13.152544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce46efbd97b04cb8894baae2e19c0972'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:13.153241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce46efbd97b04cb8894baae2e19c0972'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:13.153902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce46efbd97b04cb8894baae2e19c0972'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:13.167159 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce46efbd97b04cb8894baae2e19c0972'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:13.180229 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 921/1830] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:13 2022] DELETE /identity/v3/users/ce46efbd97b04cb8894baae2e19c0972 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:13.184792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c9db3-6b96-482b-9e67-e465b3547ef9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:13.185105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c9db3-6b96-482b-9e67-e465b3547ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:13.185379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c9db3-6b96-482b-9e67-e465b3547ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:13.203436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-807c9db3-6b96-482b-9e67-e465b3547ef9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:13.205463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-807c9db3-6b96-482b-9e67-e465b3547ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:13.284154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 910/1831] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:13.294270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:13.296001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:13.358648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363bdfd-dc4e-4cfe-a855-8450aa39b2ec admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:13.362936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299c262-f03a-4b1b-9f9c-c39467c6568d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:13.363590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c262-f03a-4b1b-9f9c-c39467c6568d service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:13.363879 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c262-f03a-4b1b-9f9c-c39467c6568d service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:13.364150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c262-f03a-4b1b-9f9c-c39467c6568d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:13.373898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299c262-f03a-4b1b-9f9c-c39467c6568d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:13.375444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299c262-f03a-4b1b-9f9c-c39467c6568d service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:13.387288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 922/1832] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:14.134770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c9db3-6b96-482b-9e67-e465b3547ef9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:14.136845 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-807c9db3-6b96-482b-9e67-e465b3547ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:14.164004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c9db3-6b96-482b-9e67-e465b3547ef9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:14.180865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8783a1-6593-4336-9fd2-d3d8a026c69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:14.182051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8783a1-6593-4336-9fd2-d3d8a026c69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:14.184596 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8783a1-6593-4336-9fd2-d3d8a026c69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:14.184885 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8783a1-6593-4336-9fd2-d3d8a026c69b admin admin] PATH_INFO: `/v3/projects/b96225d0b0e94664863b975d748d2c50` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:14.188184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8783a1-6593-4336-9fd2-d3d8a026c69b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b96225d0b0e94664863b975d748d2c50)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:14.219443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8783a1-6593-4336-9fd2-d3d8a026c69b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:14.375905 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6e8783a1-6593-4336-9fd2-d3d8a026c69b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b96225d0b0e94664863b975d748d2c50', 'request_id': 'req-6e8783a1-6593-4336-9fd2-d3d8a026c69b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:14.397497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6e8783a1-6593-4336-9fd2-d3d8a026c69b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b96225d0b0e94664863b975d748d2c50', 'request_id': 'req-6e8783a1-6593-4336-9fd2-d3d8a026c69b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:14.407475 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6e8783a1-6593-4336-9fd2-d3d8a026c69b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b96225d0b0e94664863b975d748d2c50', 'request_id': 'req-6e8783a1-6593-4336-9fd2-d3d8a026c69b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:14.434514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6e8783a1-6593-4336-9fd2-d3d8a026c69b admin admin] The token cache is being invalidate because project b96225d0b0e94664863b975d748d2c50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:14.435195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6e8783a1-6593-4336-9fd2-d3d8a026c69b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:14.439804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6e8783a1-6593-4336-9fd2-d3d8a026c69b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:14.453288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 911/1833] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:14 2022] DELETE /identity/v3/projects/b96225d0b0e94664863b975d748d2c50 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:14.459539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280e576-f943-4d6f-9367-21e25c9e17da None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:14.460023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280e576-f943-4d6f-9367-21e25c9e17da None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:14.460293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280e576-f943-4d6f-9367-21e25c9e17da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:14.494203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3280e576-f943-4d6f-9367-21e25c9e17da None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:14.495428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3280e576-f943-4d6f-9367-21e25c9e17da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:14.687273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 923/1834] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:14.695601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec291d8-3e71-4af6-acf6-9232c615f58b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:14.695887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec291d8-3e71-4af6-acf6-9232c615f58b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:14.696168 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec291d8-3e71-4af6-acf6-9232c615f58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:14.734351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bec291d8-3e71-4af6-acf6-9232c615f58b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:14.735662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bec291d8-3e71-4af6-acf6-9232c615f58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:14.835365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 912/1835] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:14.836668 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280e576-f943-4d6f-9367-21e25c9e17da None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:14.901422 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280e576-f943-4d6f-9367-21e25c9e17da None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:14.931490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebadf8-d305-4501-8106-a2cac7d6a7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:14.931811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebadf8-d305-4501-8106-a2cac7d6a7b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:14.934929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebadf8-d305-4501-8106-a2cac7d6a7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:14.934929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebadf8-d305-4501-8106-a2cac7d6a7b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:14.934929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebadf8-d305-4501-8106-a2cac7d6a7b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:14.934929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebadf8-d305-4501-8106-a2cac7d6a7b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:14.935610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebadf8-d305-4501-8106-a2cac7d6a7b8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:14.966184 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 924/1836] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:14.968133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec291d8-3e71-4af6-acf6-9232c615f58b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:14.990153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec291d8-3e71-4af6-acf6-9232c615f58b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:15.006994 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d43c36-6198-4670-87d7-2b9e24b5631c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:15.008579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d43c36-6198-4670-87d7-2b9e24b5631c admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:15.009020 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d43c36-6198-4670-87d7-2b9e24b5631c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:15.009544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d43c36-6198-4670-87d7-2b9e24b5631c admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:15.010490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d43c36-6198-4670-87d7-2b9e24b5631c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:15.013268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d43c36-6198-4670-87d7-2b9e24b5631c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:15.022095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbc3ce-c4cc-4033-8853-0a3b9a90fa70 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:15.022743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbc3ce-c4cc-4033-8853-0a3b9a90fa70 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:15.024388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbc3ce-c4cc-4033-8853-0a3b9a90fa70 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:15.026028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 925/1837] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:18:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:18:15.072234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:15.089829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de826161-c68c-432d-b966-779e1caaddcf None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:15.090440 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de826161-c68c-432d-b966-779e1caaddcf None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:15.090606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de826161-c68c-432d-b966-779e1caaddcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:15.150701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 913/1838] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:14 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:15.151763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d43c36-6198-4670-87d7-2b9e24b5631c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:15.161656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-de826161-c68c-432d-b966-779e1caaddcf None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:15.165485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-de826161-c68c-432d-b966-779e1caaddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:15.219742 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:15.298293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d43c36-6198-4670-87d7-2b9e24b5631c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:15.307535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9059813-8438-44cb-8880-a56abdcc0415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:15.310998 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9059813-8438-44cb-8880-a56abdcc0415 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:15.311389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9059813-8438-44cb-8880-a56abdcc0415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:15.312432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9059813-8438-44cb-8880-a56abdcc0415 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:15.315882 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9059813-8438-44cb-8880-a56abdcc0415 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:15.324270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9059813-8438-44cb-8880-a56abdcc0415 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:15.339024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 926/1839] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:18:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:15.367473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 914/1840] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:15 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:15.368641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de826161-c68c-432d-b966-779e1caaddcf None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:15.385126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de826161-c68c-432d-b966-779e1caaddcf None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:15.394345 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a52588-47e5-48a1-b742-9ccae2b6e7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:15.396573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a52588-47e5-48a1-b742-9ccae2b6e7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:15.397059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a52588-47e5-48a1-b742-9ccae2b6e7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:15.399799 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a52588-47e5-48a1-b742-9ccae2b6e7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:15.401121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a52588-47e5-48a1-b742-9ccae2b6e7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:15.403357 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a52588-47e5-48a1-b742-9ccae2b6e7c6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:15.430833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 927/1841] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:15.432127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9059813-8438-44cb-8880-a56abdcc0415 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:15.461981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9059813-8438-44cb-8880-a56abdcc0415 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:15.481652 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be93160-457a-4dcd-885e-dcec33a307ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:15.482364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be93160-457a-4dcd-885e-dcec33a307ec admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:15.482646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be93160-457a-4dcd-885e-dcec33a307ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:15.483003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be93160-457a-4dcd-885e-dcec33a307ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:15.484147 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a52588-47e5-48a1-b742-9ccae2b6e7c6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:15.484672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be93160-457a-4dcd-885e-dcec33a307ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:15.486719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-66a52588-47e5-48a1-b742-9ccae2b6e7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:15.490580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be93160-457a-4dcd-885e-dcec33a307ec admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:15.519722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 915/1842] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:15.524516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be93160-457a-4dcd-885e-dcec33a307ec admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:15.528848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a52588-47e5-48a1-b742-9ccae2b6e7c6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:15.547670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e2c3f-db39-466f-9058-0f5cf07b5467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:15.552240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e2c3f-db39-466f-9058-0f5cf07b5467 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:15.553427 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e2c3f-db39-466f-9058-0f5cf07b5467 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:15.554092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e2c3f-db39-466f-9058-0f5cf07b5467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:15.554415 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be93160-457a-4dcd-885e-dcec33a307ec admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:15.567959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b477378-1330-4764-8329-66ae5b4a5264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:15.569254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b477378-1330-4764-8329-66ae5b4a5264 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:15.570497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b477378-1330-4764-8329-66ae5b4a5264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:15.570981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b477378-1330-4764-8329-66ae5b4a5264 admin admin] PATH_INFO: `/v3/projects/193e991a3697426e9663f7e8e136ce05/users/e9057d1866f649df9963f66bcedda2e5/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:15.594030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e2c3f-db39-466f-9058-0f5cf07b5467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:15.598440 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e2c3f-db39-466f-9058-0f5cf07b5467 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:15.614156 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b477378-1330-4764-8329-66ae5b4a5264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=193e991a3697426e9663f7e8e136ce05, user_id=e9057d1866f649df9963f66bcedda2e5, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:15.619728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b477378-1330-4764-8329-66ae5b4a5264 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:15.647365 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 928/1843] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:15.651607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 916/1844] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:15 2022] PUT /identity/v3/projects/193e991a3697426e9663f7e8e136ce05/users/e9057d1866f649df9963f66bcedda2e5/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:15.670334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9a5ae-3836-490e-b2a4-a6128f9c57c8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:15.671549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9a5ae-3836-490e-b2a4-a6128f9c57c8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:15.672012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9a5ae-3836-490e-b2a4-a6128f9c57c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:15.741127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-34a9a5ae-3836-490e-b2a4-a6128f9c57c8 None None] MFA Rules not processed for user `e9057d1866f649df9963f66bcedda2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:15.742577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-34a9a5ae-3836-490e-b2a4-a6128f9c57c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:15.892770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 929/1845] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:15.897758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b477378-1330-4764-8329-66ae5b4a5264 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:16.062637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b477378-1330-4764-8329-66ae5b4a5264 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:16.082875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb56a5-991c-49ea-bc9c-4407dd8e2ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:16.084355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb56a5-991c-49ea-bc9c-4407dd8e2ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:16.084355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb56a5-991c-49ea-bc9c-4407dd8e2ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:16.084551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb56a5-991c-49ea-bc9c-4407dd8e2ea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:16.085471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb56a5-991c-49ea-bc9c-4407dd8e2ea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:16.095787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb56a5-991c-49ea-bc9c-4407dd8e2ea4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:16.141879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 917/1846] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:15 2022] POST /identity/v3/projects => generated 377 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:16.151192 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9a5ae-3836-490e-b2a4-a6128f9c57c8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:16.172299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9a5ae-3836-490e-b2a4-a6128f9c57c8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:16.363657 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a94a6-7878-461a-a4ac-6d6d9ae795fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:16.364358 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a94a6-7878-461a-a4ac-6d6d9ae795fb admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:16.364691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a94a6-7878-461a-a4ac-6d6d9ae795fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:16.365268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a94a6-7878-461a-a4ac-6d6d9ae795fb admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:16.366798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a94a6-7878-461a-a4ac-6d6d9ae795fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:16.369985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a94a6-7878-461a-a4ac-6d6d9ae795fb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:16.410227 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 930/1847] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:16 2022] POST /identity/v3/users => generated 289 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:16.411619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb56a5-991c-49ea-bc9c-4407dd8e2ea4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:16.428419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb56a5-991c-49ea-bc9c-4407dd8e2ea4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:16.444361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8d3f3-36f4-4401-9af9-31edea4b99eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:16.444951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8d3f3-36f4-4401-9af9-31edea4b99eb admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:16.445389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8d3f3-36f4-4401-9af9-31edea4b99eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:16.445675 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8d3f3-36f4-4401-9af9-31edea4b99eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:16.446496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8d3f3-36f4-4401-9af9-31edea4b99eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:16.448216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8d3f3-36f4-4401-9af9-31edea4b99eb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:16.467172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 918/1848] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:16.468664 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a94a6-7878-461a-a4ac-6d6d9ae795fb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:16.479360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:16.485796 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a94a6-7878-461a-a4ac-6d6d9ae795fb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:16.497981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7e581-b7aa-4b41-9788-95906ea2e859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:16.501364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7e581-b7aa-4b41-9788-95906ea2e859 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:16.503516 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7e581-b7aa-4b41-9788-95906ea2e859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:16.504001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7e581-b7aa-4b41-9788-95906ea2e859 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:16.505117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7e581-b7aa-4b41-9788-95906ea2e859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:16.516242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7e581-b7aa-4b41-9788-95906ea2e859 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:16.552460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8d3f3-36f4-4401-9af9-31edea4b99eb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:16.553556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 931/1849] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:16.568806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8d3f3-36f4-4401-9af9-31edea4b99eb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:16.586926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470ac77-5146-40b3-b34a-f4a57385d093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:16.589758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470ac77-5146-40b3-b34a-f4a57385d093 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:16.591619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470ac77-5146-40b3-b34a-f4a57385d093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:16.592099 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470ac77-5146-40b3-b34a-f4a57385d093 admin admin] PATH_INFO: `/v3/projects/7620cf483bfc45658d659e77d41d28ff/users/63565d9a97514133ae4ee38043695d3e/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:16.627436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470ac77-5146-40b3-b34a-f4a57385d093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7620cf483bfc45658d659e77d41d28ff, user_id=63565d9a97514133ae4ee38043695d3e, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:16.633987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470ac77-5146-40b3-b34a-f4a57385d093 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:16.639749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:16.662709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 919/1850] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:16 2022] PUT /identity/v3/projects/7620cf483bfc45658d659e77d41d28ff/users/63565d9a97514133ae4ee38043695d3e/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:16.681270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15190a3a-90b1-4a77-88f4-656bf709d204 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:16.681873 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15190a3a-90b1-4a77-88f4-656bf709d204 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:16.682325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15190a3a-90b1-4a77-88f4-656bf709d204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:16.718515 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-15190a3a-90b1-4a77-88f4-656bf709d204 None None] MFA Rules not processed for user `63565d9a97514133ae4ee38043695d3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:16.720686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-15190a3a-90b1-4a77-88f4-656bf709d204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:16.825252 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 932/1851] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:16.830822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470ac77-5146-40b3-b34a-f4a57385d093 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:16.905794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470ac77-5146-40b3-b34a-f4a57385d093 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:16.910644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218082e-8ef1-4b19-be0c-146e5624975a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:16.911310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218082e-8ef1-4b19-be0c-146e5624975a admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:16.911590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218082e-8ef1-4b19-be0c-146e5624975a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:16.911868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218082e-8ef1-4b19-be0c-146e5624975a admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:16.912555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218082e-8ef1-4b19-be0c-146e5624975a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:16.914624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218082e-8ef1-4b19-be0c-146e5624975a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:16.944368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 920/1852] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:16 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:16.948209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15190a3a-90b1-4a77-88f4-656bf709d204 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:16.964076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15190a3a-90b1-4a77-88f4-656bf709d204 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:17.033820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04073c4f-8fa8-48d8-a071-25e1ac50a27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:17.035862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04073c4f-8fa8-48d8-a071-25e1ac50a27f admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:17.036167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04073c4f-8fa8-48d8-a071-25e1ac50a27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:17.036487 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04073c4f-8fa8-48d8-a071-25e1ac50a27f admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:17.037193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04073c4f-8fa8-48d8-a071-25e1ac50a27f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:17.039226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04073c4f-8fa8-48d8-a071-25e1ac50a27f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:17.062098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 933/1853] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:16 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:17.066833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218082e-8ef1-4b19-be0c-146e5624975a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:17.078141 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218082e-8ef1-4b19-be0c-146e5624975a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:17.083699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc2878-5d39-4c3c-9f72-ceadf22bab1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:17.084767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc2878-5d39-4c3c-9f72-ceadf22bab1e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:17.085101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc2878-5d39-4c3c-9f72-ceadf22bab1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:17.085411 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc2878-5d39-4c3c-9f72-ceadf22bab1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:17.086290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc2878-5d39-4c3c-9f72-ceadf22bab1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:17.088310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc2878-5d39-4c3c-9f72-ceadf22bab1e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:17.103834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 921/1854] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:17.108234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04073c4f-8fa8-48d8-a071-25e1ac50a27f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:17.121441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04073c4f-8fa8-48d8-a071-25e1ac50a27f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:17.126449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03af99-5cf7-41b4-841f-a3efde05afb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:17.128107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03af99-5cf7-41b4-841f-a3efde05afb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:17.128550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03af99-5cf7-41b4-841f-a3efde05afb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:17.128836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03af99-5cf7-41b4-841f-a3efde05afb4 admin admin] PATH_INFO: `/v3/projects/966fa72eb0634a4f95f16fb9a96edc9e/users/7a0f3312bb2c4b2789e4cd6755e820a9/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:17.144893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03af99-5cf7-41b4-841f-a3efde05afb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966fa72eb0634a4f95f16fb9a96edc9e, user_id=7a0f3312bb2c4b2789e4cd6755e820a9, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:17.147640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03af99-5cf7-41b4-841f-a3efde05afb4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:17.163782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 934/1855] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:17 2022] PUT /identity/v3/projects/966fa72eb0634a4f95f16fb9a96edc9e/users/7a0f3312bb2c4b2789e4cd6755e820a9/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:17.167902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc2878-5d39-4c3c-9f72-ceadf22bab1e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:17.178393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc2878-5d39-4c3c-9f72-ceadf22bab1e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:17.246497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6f1cd-9f63-4682-bbe1-e7f7db13d1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:17.247645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6f1cd-9f63-4682-bbe1-e7f7db13d1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:17.248078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6f1cd-9f63-4682-bbe1-e7f7db13d1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:17.248520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6f1cd-9f63-4682-bbe1-e7f7db13d1a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:17.249605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6f1cd-9f63-4682-bbe1-e7f7db13d1a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:17.261681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6f1cd-9f63-4682-bbe1-e7f7db13d1a0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:17.279860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 922/1856] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:17.285116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03af99-5cf7-41b4-841f-a3efde05afb4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:17.296789 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03af99-5cf7-41b4-841f-a3efde05afb4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:17.302100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2346853-f27c-4896-aff4-f3c558c0a18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:17.303040 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2346853-f27c-4896-aff4-f3c558c0a18e admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:17.303353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2346853-f27c-4896-aff4-f3c558c0a18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:17.303599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2346853-f27c-4896-aff4-f3c558c0a18e admin admin] PATH_INFO: `/v3/projects/966fa72eb0634a4f95f16fb9a96edc9e/users/7a0f3312bb2c4b2789e4cd6755e820a9/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:17.305624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2346853-f27c-4896-aff4-f3c558c0a18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=966fa72eb0634a4f95f16fb9a96edc9e, user_id=7a0f3312bb2c4b2789e4cd6755e820a9, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:17.308353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2346853-f27c-4896-aff4-f3c558c0a18e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:17.325583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 935/1857] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:17 2022] PUT /identity/v3/projects/966fa72eb0634a4f95f16fb9a96edc9e/users/7a0f3312bb2c4b2789e4cd6755e820a9/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:17.336324 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6254fa0-4ebf-4b4d-b88f-39e2c91c2d2d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:17.336892 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6254fa0-4ebf-4b4d-b88f-39e2c91c2d2d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:17.337374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6254fa0-4ebf-4b4d-b88f-39e2c91c2d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:17.421557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e6254fa0-4ebf-4b4d-b88f-39e2c91c2d2d None None] MFA Rules not processed for user `7a0f3312bb2c4b2789e4cd6755e820a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:17.425673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e6254fa0-4ebf-4b4d-b88f-39e2c91c2d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:17.469196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2346853-f27c-4896-aff4-f3c558c0a18e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:17.479642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f2346853-f27c-4896-aff4-f3c558c0a18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:17.502248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:17.552003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 923/1858] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:17.560469 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:17.568205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6254fa0-4ebf-4b4d-b88f-39e2c91c2d2d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:17.572704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e6254fa0-4ebf-4b4d-b88f-39e2c91c2d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:17.627754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2346853-f27c-4896-aff4-f3c558c0a18e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:17.641113 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:17.642992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:17.643426 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:17.643862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] PATH_INFO: `/v3/users/e73264257ac541269c1685b5355d4d1c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:17.648209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e73264257ac541269c1685b5355d4d1c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:17.650234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:17.715876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6254fa0-4ebf-4b4d-b88f-39e2c91c2d2d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:17.722338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a02619-dd9f-4015-b97e-299cdbc53a8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:17.724005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a02619-dd9f-4015-b97e-299cdbc53a8d service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:17.724617 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a02619-dd9f-4015-b97e-299cdbc53a8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:17.725316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a02619-dd9f-4015-b97e-299cdbc53a8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:17.767962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e73264257ac541269c1685b5355d4d1c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.786424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e73264257ac541269c1685b5355d4d1c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.787211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e73264257ac541269c1685b5355d4d1c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.787939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e73264257ac541269c1685b5355d4d1c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.828695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e73264257ac541269c1685b5355d4d1c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.833586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a02619-dd9f-4015-b97e-299cdbc53a8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:17.836034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a02619-dd9f-4015-b97e-299cdbc53a8d service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:17.844240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 936/1859] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:17 2022] DELETE /identity/v3/users/e73264257ac541269c1685b5355d4d1c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:17.845614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:17.847994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:17.851970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 924/1860] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:17.853294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e3862-2b04-4db1-a41c-5ef7f718d6a1 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:17.853582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e3862-2b04-4db1-a41c-5ef7f718d6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:17.853888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e3862-2b04-4db1-a41c-5ef7f718d6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:17.901551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-650e3862-2b04-4db1-a41c-5ef7f718d6a1 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:17.902861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-650e3862-2b04-4db1-a41c-5ef7f718d6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:17.971246 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd5ac8-a792-47b4-9282-dea46747d1b9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:17.976245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07473f6-b425-4836-8bc2-3e6389787516 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:17.977193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07473f6-b425-4836-8bc2-3e6389787516 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:17.978374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07473f6-b425-4836-8bc2-3e6389787516 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:17.978750 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07473f6-b425-4836-8bc2-3e6389787516 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:17.981524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a07473f6-b425-4836-8bc2-3e6389787516 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:18.039676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 925/1861] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:18.054808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e3862-2b04-4db1-a41c-5ef7f718d6a1 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:18.056629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-650e3862-2b04-4db1-a41c-5ef7f718d6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:18.108944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07473f6-b425-4836-8bc2-3e6389787516 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:18.110444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07473f6-b425-4836-8bc2-3e6389787516 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:18.134050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 937/1862] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:18.161690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:18.171539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e3862-2b04-4db1-a41c-5ef7f718d6a1 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:18.178618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc41efb-3019-45b3-943e-f194fcdad878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:18.179299 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc41efb-3019-45b3-943e-f194fcdad878 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:18.179581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc41efb-3019-45b3-943e-f194fcdad878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:18.179859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc41efb-3019-45b3-943e-f194fcdad878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:18.194654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc41efb-3019-45b3-943e-f194fcdad878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:18.196534 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc41efb-3019-45b3-943e-f194fcdad878 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:18.215035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 926/1863] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:18.834855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07473f6-b425-4836-8bc2-3e6389787516 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:18.837645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a07473f6-b425-4836-8bc2-3e6389787516 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:18.862089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07473f6-b425-4836-8bc2-3e6389787516 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:18.870499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:18.871701 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:18.871989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:18.872327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] PATH_INFO: `/v3/projects/ab2f116ff48f4fa38a88217603c4500a` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:18.874631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab2f116ff48f4fa38a88217603c4500a)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:18.877801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:18.880509 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:18.978724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab2f116ff48f4fa38a88217603c4500a', 'request_id': 'req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:18.991126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab2f116ff48f4fa38a88217603c4500a', 'request_id': 'req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.008670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab2f116ff48f4fa38a88217603c4500a', 'request_id': 'req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.045206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] The token cache is being invalidate because project ab2f116ff48f4fa38a88217603c4500a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:19.045504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.048219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.050488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 938/1864] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:18 2022] DELETE /identity/v3/projects/ab2f116ff48f4fa38a88217603c4500a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:19.055862 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc41efb-3019-45b3-943e-f194fcdad878 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:19.075648 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc41efb-3019-45b3-943e-f194fcdad878 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:19.161665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:19.162815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:19.163196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:19.163674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] PATH_INFO: `/v3/users/8597d4ac1d904368a237032721543113` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:19.165022 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8597d4ac1d904368a237032721543113)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:19.167223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:19.240995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8597d4ac1d904368a237032721543113'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.260283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8597d4ac1d904368a237032721543113'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.277569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8597d4ac1d904368a237032721543113'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.278629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8597d4ac1d904368a237032721543113'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.287924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8597d4ac1d904368a237032721543113'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.299365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 927/1865] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:19 2022] DELETE /identity/v3/users/8597d4ac1d904368a237032721543113 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:19.688085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:19.693167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:19.714847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8d4d6-c8ce-48e4-95b9-38abd41756a9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:19.807274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33da0c1-52aa-470f-9f93-640b0b635e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:19.808479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33da0c1-52aa-470f-9f93-640b0b635e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:19.809079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33da0c1-52aa-470f-9f93-640b0b635e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:19.810192 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33da0c1-52aa-470f-9f93-640b0b635e87 admin admin] PATH_INFO: `/v3/projects/b771aef331264e51bf044e5ce47ee693` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:19.812402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33da0c1-52aa-470f-9f93-640b0b635e87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b771aef331264e51bf044e5ce47ee693)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:19.815756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33da0c1-52aa-470f-9f93-640b0b635e87 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:19.837357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:19.839186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:19.920090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e33da0c1-52aa-470f-9f93-640b0b635e87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b771aef331264e51bf044e5ce47ee693', 'request_id': 'req-e33da0c1-52aa-470f-9f93-640b0b635e87'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.940676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e33da0c1-52aa-470f-9f93-640b0b635e87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b771aef331264e51bf044e5ce47ee693', 'request_id': 'req-e33da0c1-52aa-470f-9f93-640b0b635e87'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.952692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e33da0c1-52aa-470f-9f93-640b0b635e87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b771aef331264e51bf044e5ce47ee693', 'request_id': 'req-e33da0c1-52aa-470f-9f93-640b0b635e87'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.969650 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff9ea2-1a20-4697-91fd-577c704ae15c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:19.982037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e33da0c1-52aa-470f-9f93-640b0b635e87 admin admin] The token cache is being invalidate because project b771aef331264e51bf044e5ce47ee693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:19.982600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e33da0c1-52aa-470f-9f93-640b0b635e87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.985067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e33da0c1-52aa-470f-9f93-640b0b635e87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:19.988244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 939/1866] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:19 2022] DELETE /identity/v3/projects/b771aef331264e51bf044e5ce47ee693 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:20.001766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffdc37-9166-4409-b50c-ddcfeb6b61c3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.002046 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffdc37-9166-4409-b50c-ddcfeb6b61c3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.002347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffdc37-9166-4409-b50c-ddcfeb6b61c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.022294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:20.070737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-daffdc37-9166-4409-b50c-ddcfeb6b61c3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:20.072017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-daffdc37-9166-4409-b50c-ddcfeb6b61c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:20.072843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0981cb5-755a-4a3a-88ac-b9a2386b8cf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:20.074204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0981cb5-755a-4a3a-88ac-b9a2386b8cf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.075154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0981cb5-755a-4a3a-88ac-b9a2386b8cf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.075627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0981cb5-755a-4a3a-88ac-b9a2386b8cf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.080571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a0981cb5-755a-4a3a-88ac-b9a2386b8cf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:20.120096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:20.192232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 940/1867] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:20.194025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d8589-bbe7-42b2-9d59-c9578abe624d None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.199198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d8589-bbe7-42b2-9d59-c9578abe624d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.199670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d8589-bbe7-42b2-9d59-c9578abe624d None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.201346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 941/1868] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:18:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:18:20.203276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a35c5-4970-42ee-8dbd-9b46444ca5cb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.205749 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a35c5-4970-42ee-8dbd-9b46444ca5cb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.206235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a35c5-4970-42ee-8dbd-9b46444ca5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.211051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0981cb5-755a-4a3a-88ac-b9a2386b8cf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:20.213335 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0981cb5-755a-4a3a-88ac-b9a2386b8cf1 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:20.240106 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 928/1869] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:20.243464 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd78dc1-c192-4aa7-8b76-4cb56b5183ff None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.244128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd78dc1-c192-4aa7-8b76-4cb56b5183ff None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.245640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd78dc1-c192-4aa7-8b76-4cb56b5183ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.254689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f62a35c5-4970-42ee-8dbd-9b46444ca5cb None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:20.259807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f62a35c5-4970-42ee-8dbd-9b46444ca5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:20.280890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 942/1870] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:20.286855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a35c5-4970-42ee-8dbd-9b46444ca5cb None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:20.294283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ccd78dc1-c192-4aa7-8b76-4cb56b5183ff None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:20.297209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ccd78dc1-c192-4aa7-8b76-4cb56b5183ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:20.305156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a35c5-4970-42ee-8dbd-9b46444ca5cb None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:20.319737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfec2fb-20ff-4b13-b5c2-9fc74cc10b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:20.320596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfec2fb-20ff-4b13-b5c2-9fc74cc10b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.320891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfec2fb-20ff-4b13-b5c2-9fc74cc10b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.321170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfec2fb-20ff-4b13-b5c2-9fc74cc10b4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.321957 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfec2fb-20ff-4b13-b5c2-9fc74cc10b4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:20.324656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfec2fb-20ff-4b13-b5c2-9fc74cc10b4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:20.326301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfec2fb-20ff-4b13-b5c2-9fc74cc10b4b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:20.367052 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 943/1871] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:18:20.377286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfec2fb-20ff-4b13-b5c2-9fc74cc10b4b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:20.403996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfec2fb-20ff-4b13-b5c2-9fc74cc10b4b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:20.423749 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72308798-15a7-48d0-bf51-0eba4344abcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:20.424857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72308798-15a7-48d0-bf51-0eba4344abcb admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.425425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72308798-15a7-48d0-bf51-0eba4344abcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.425876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72308798-15a7-48d0-bf51-0eba4344abcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.426791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72308798-15a7-48d0-bf51-0eba4344abcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:20.431114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72308798-15a7-48d0-bf51-0eba4344abcb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:20.501824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 944/1872] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:20 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:20.506044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72308798-15a7-48d0-bf51-0eba4344abcb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:20.511582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 929/1873] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:18:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:20.547060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd78dc1-c192-4aa7-8b76-4cb56b5183ff None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:20.551855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ccd78dc1-c192-4aa7-8b76-4cb56b5183ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:20.596110 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd78dc1-c192-4aa7-8b76-4cb56b5183ff None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:20.603524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72308798-15a7-48d0-bf51-0eba4344abcb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:20.609721 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1973a0-adff-4a06-84c1-6bc73c11ec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:20.610601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1973a0-adff-4a06-84c1-6bc73c11ec2a admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.611435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1973a0-adff-4a06-84c1-6bc73c11ec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.611684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1973a0-adff-4a06-84c1-6bc73c11ec2a admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.613495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1973a0-adff-4a06-84c1-6bc73c11ec2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:20.615747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1973a0-adff-4a06-84c1-6bc73c11ec2a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:20.655981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 945/1874] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:20 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:20.658059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1973a0-adff-4a06-84c1-6bc73c11ec2a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:20.675124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1973a0-adff-4a06-84c1-6bc73c11ec2a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:20.683072 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e2251-3509-45a2-967e-b3ca70ddf945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:20.684707 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2251-3509-45a2-967e-b3ca70ddf945 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.685998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2251-3509-45a2-967e-b3ca70ddf945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.686892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2251-3509-45a2-967e-b3ca70ddf945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.687456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e2251-3509-45a2-967e-b3ca70ddf945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:20.690653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e2251-3509-45a2-967e-b3ca70ddf945 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:20.714340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 946/1875] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:20.715669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e2251-3509-45a2-967e-b3ca70ddf945 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:20.742473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e2251-3509-45a2-967e-b3ca70ddf945 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:20.752350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90430338-a88d-483a-bdc9-60444b508232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:20.753454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90430338-a88d-483a-bdc9-60444b508232 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.754002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90430338-a88d-483a-bdc9-60444b508232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.754242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90430338-a88d-483a-bdc9-60444b508232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.754988 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90430338-a88d-483a-bdc9-60444b508232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:20.757034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90430338-a88d-483a-bdc9-60444b508232 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:20.774177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85763ab3-c99b-42b4-8246-7a00acb20240 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:20.774850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85763ab3-c99b-42b4-8246-7a00acb20240 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.775132 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85763ab3-c99b-42b4-8246-7a00acb20240 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.775416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85763ab3-c99b-42b4-8246-7a00acb20240 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.776860 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 947/1876] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:20.781587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90430338-a88d-483a-bdc9-60444b508232 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:20.789333 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:20.803019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90430338-a88d-483a-bdc9-60444b508232 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:20.803674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85763ab3-c99b-42b4-8246-7a00acb20240 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:20.805674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85763ab3-c99b-42b4-8246-7a00acb20240 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:20.808242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985dbade-28a2-4983-9e66-b60bf69c4895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:20.809135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985dbade-28a2-4983-9e66-b60bf69c4895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.809420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985dbade-28a2-4983-9e66-b60bf69c4895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.810108 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985dbade-28a2-4983-9e66-b60bf69c4895 admin admin] PATH_INFO: `/v3/projects/cfb4559124a04485a412722864f5dc5f/users/7c394836f8214070b0eda102a6cc9cd4/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.831408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985dbade-28a2-4983-9e66-b60bf69c4895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb4559124a04485a412722864f5dc5f, user_id=7c394836f8214070b0eda102a6cc9cd4, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:20.834850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985dbade-28a2-4983-9e66-b60bf69c4895 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:20.847642 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 930/1877] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:18:20.855368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 948/1878] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:20 2022] PUT /identity/v3/projects/cfb4559124a04485a412722864f5dc5f/users/7c394836f8214070b0eda102a6cc9cd4/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:20.861424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05800e78-c7f5-4fd1-aba9-ff5efc7cfd46 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:20.861705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05800e78-c7f5-4fd1-aba9-ff5efc7cfd46 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:20.862020 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05800e78-c7f5-4fd1-aba9-ff5efc7cfd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:20.926005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-05800e78-c7f5-4fd1-aba9-ff5efc7cfd46 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:20.928877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-05800e78-c7f5-4fd1-aba9-ff5efc7cfd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:21.097778 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 931/1879] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:21.108606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985dbade-28a2-4983-9e66-b60bf69c4895 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:21.111468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-985dbade-28a2-4983-9e66-b60bf69c4895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:21.192390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:21.262132 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985dbade-28a2-4983-9e66-b60bf69c4895 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:21.268457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14a91f-742e-42db-b894-f08245220d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:21.269224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14a91f-742e-42db-b894-f08245220d5b service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:21.269513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14a91f-742e-42db-b894-f08245220d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:21.269977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14a91f-742e-42db-b894-f08245220d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:21.285096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14a91f-742e-42db-b894-f08245220d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:21.286748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14a91f-742e-42db-b894-f08245220d5b service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:21.310597 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 949/1880] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:21.445809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05800e78-c7f5-4fd1-aba9-ff5efc7cfd46 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:21.447293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-05800e78-c7f5-4fd1-aba9-ff5efc7cfd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:21.470139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14a91f-742e-42db-b894-f08245220d5b service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:21.471895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ac14a91f-742e-42db-b894-f08245220d5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:21.503887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14a91f-742e-42db-b894-f08245220d5b service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:21.516209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe21502-7702-456b-b1b1-d59bbaf9f29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:21.519595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe21502-7702-456b-b1b1-d59bbaf9f29c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:21.520790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe21502-7702-456b-b1b1-d59bbaf9f29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:21.521088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe21502-7702-456b-b1b1-d59bbaf9f29c admin admin] PATH_INFO: `/v3/users/96ff01b0d0d64d05af2d8fbefa129bfa` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:21.524002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe21502-7702-456b-b1b1-d59bbaf9f29c admin admin] RBAC: Authorizing `identity:delete_user(user_id=96ff01b0d0d64d05af2d8fbefa129bfa)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:21.529189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe21502-7702-456b-b1b1-d59bbaf9f29c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:21.575115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05800e78-c7f5-4fd1-aba9-ff5efc7cfd46 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:21.579302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a475569-6b98-4a9d-b34e-2835665da370 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:21.580629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a475569-6b98-4a9d-b34e-2835665da370 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:21.580759 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a475569-6b98-4a9d-b34e-2835665da370 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:21.581019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a475569-6b98-4a9d-b34e-2835665da370 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:21.587130 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3a475569-6b98-4a9d-b34e-2835665da370 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:21.648964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1fe21502-7702-456b-b1b1-d59bbaf9f29c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96ff01b0d0d64d05af2d8fbefa129bfa'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:21.661010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1fe21502-7702-456b-b1b1-d59bbaf9f29c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96ff01b0d0d64d05af2d8fbefa129bfa'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:21.662633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1fe21502-7702-456b-b1b1-d59bbaf9f29c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96ff01b0d0d64d05af2d8fbefa129bfa'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:21.669133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1fe21502-7702-456b-b1b1-d59bbaf9f29c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96ff01b0d0d64d05af2d8fbefa129bfa'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:21.702060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1fe21502-7702-456b-b1b1-d59bbaf9f29c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96ff01b0d0d64d05af2d8fbefa129bfa'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:21.728021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 950/1881] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:21 2022] DELETE /identity/v3/users/96ff01b0d0d64d05af2d8fbefa129bfa => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:21.734726 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b5d5b-403a-47ad-9a49-2241cec10f10 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:21.735022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b5d5b-403a-47ad-9a49-2241cec10f10 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:21.735305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b5d5b-403a-47ad-9a49-2241cec10f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:21.764797 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a475569-6b98-4a9d-b34e-2835665da370 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:21.766501 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a475569-6b98-4a9d-b34e-2835665da370 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:21.786679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 932/1882] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:21.805498 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-109b5d5b-403a-47ad-9a49-2241cec10f10 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:21.806812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-109b5d5b-403a-47ad-9a49-2241cec10f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:21.973335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 951/1883] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:22.042312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a475569-6b98-4a9d-b34e-2835665da370 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:22.059963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a475569-6b98-4a9d-b34e-2835665da370 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:22.166556 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:22.180026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea5bac-3be2-4180-8129-3129a7f98629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:22.181191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea5bac-3be2-4180-8129-3129a7f98629 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:22.181624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea5bac-3be2-4180-8129-3129a7f98629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:22.182095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea5bac-3be2-4180-8129-3129a7f98629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:22.200885 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea5bac-3be2-4180-8129-3129a7f98629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:22.202760 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea5bac-3be2-4180-8129-3129a7f98629 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:22.217667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 933/1884] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:22.662396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64372045-ab74-4949-ad66-76549362c5d4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:22.662956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64372045-ab74-4949-ad66-76549362c5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:22.663488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64372045-ab74-4949-ad66-76549362c5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:22.726146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-64372045-ab74-4949-ad66-76549362c5d4 None None] MFA Rules not processed for user `7c394836f8214070b0eda102a6cc9cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:22.727735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-64372045-ab74-4949-ad66-76549362c5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:22.776677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:22.826629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea5bac-3be2-4180-8129-3129a7f98629 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:22.828956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-63ea5bac-3be2-4180-8129-3129a7f98629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:22.861144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea5bac-3be2-4180-8129-3129a7f98629 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:22.862886 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 952/1885] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:22.866940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64372045-ab74-4949-ad66-76549362c5d4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:22.868725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-64372045-ab74-4949-ad66-76549362c5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:22.875696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:22.877608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:22.878108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:22.878530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] PATH_INFO: `/v3/projects/b906dcf1af434eb79ee544471ee21382` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:22.880535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b906dcf1af434eb79ee544471ee21382)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:22.890969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64372045-ab74-4949-ad66-76549362c5d4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:22.894901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:22.898712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3f157-d6a9-406f-a7b3-0d873f64c82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:22.900001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3f157-d6a9-406f-a7b3-0d873f64c82c admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:22.900676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3f157-d6a9-406f-a7b3-0d873f64c82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:22.901241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3f157-d6a9-406f-a7b3-0d873f64c82c admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:22.902194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3f157-d6a9-406f-a7b3-0d873f64c82c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:22.904851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3f157-d6a9-406f-a7b3-0d873f64c82c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:22.950299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 953/1886] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:22 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:22.955326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3f157-d6a9-406f-a7b3-0d873f64c82c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:22.981600 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b906dcf1af434eb79ee544471ee21382', 'request_id': 'req-bf739c82-db96-47c5-8a1a-28a42873135b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:22.995146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b906dcf1af434eb79ee544471ee21382', 'request_id': 'req-bf739c82-db96-47c5-8a1a-28a42873135b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:23.012329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b906dcf1af434eb79ee544471ee21382', 'request_id': 'req-bf739c82-db96-47c5-8a1a-28a42873135b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:23.032718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] The token cache is being invalidate because project b906dcf1af434eb79ee544471ee21382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:23.033185 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:23.034064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:23.039336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 934/1887] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:22 2022] DELETE /identity/v3/projects/b906dcf1af434eb79ee544471ee21382 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:23.041255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:23.044906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:23.090795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3f157-d6a9-406f-a7b3-0d873f64c82c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:23.179459 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf739c82-db96-47c5-8a1a-28a42873135b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:23.196787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:23.197678 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:23.198500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:23.198777 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] PATH_INFO: `/v3/users/3f9f303710b848178db0902cd23e9601` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:23.203305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d352ddb-59ed-4614-9252-beed506b309a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:23.206144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f9f303710b848178db0902cd23e9601)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:23.206273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d352ddb-59ed-4614-9252-beed506b309a admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:23.206498 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d352ddb-59ed-4614-9252-beed506b309a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:23.206780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d352ddb-59ed-4614-9252-beed506b309a admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:23.207911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d352ddb-59ed-4614-9252-beed506b309a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:23.210856 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d352ddb-59ed-4614-9252-beed506b309a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:23.211478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:23.248312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 954/1888] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:22 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:23.252041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d352ddb-59ed-4614-9252-beed506b309a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:23.255896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8d352ddb-59ed-4614-9252-beed506b309a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:23.290418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d352ddb-59ed-4614-9252-beed506b309a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:23.302292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8210cb-c582-413d-b8d5-04a515eaebe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:23.303034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8210cb-c582-413d-b8d5-04a515eaebe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:23.304179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8210cb-c582-413d-b8d5-04a515eaebe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:23.304531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8210cb-c582-413d-b8d5-04a515eaebe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:23.307569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8210cb-c582-413d-b8d5-04a515eaebe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:23.311667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8210cb-c582-413d-b8d5-04a515eaebe3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:23.325010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9f303710b848178db0902cd23e9601'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:23.335386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 955/1889] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:23.340962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8210cb-c582-413d-b8d5-04a515eaebe3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:23.342202 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9f303710b848178db0902cd23e9601'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:23.366013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8210cb-c582-413d-b8d5-04a515eaebe3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:23.382775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9f303710b848178db0902cd23e9601'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:23.382775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9f303710b848178db0902cd23e9601'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:23.382775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:23.392620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f77120-3ada-42bf-9c79-22afa5d0c4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:23.395938 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f77120-3ada-42bf-9c79-22afa5d0c4fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:23.399276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f77120-3ada-42bf-9c79-22afa5d0c4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:23.399408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9f303710b848178db0902cd23e9601'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:23.399829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f77120-3ada-42bf-9c79-22afa5d0c4fd admin admin] PATH_INFO: `/v3/projects/db7cf079fb8e4910afa6e4639616bef9/users/d93bf5ee781440c8806d4f77d9d45afc/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:23.418521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 935/1890] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:23 2022] DELETE /identity/v3/users/3f9f303710b848178db0902cd23e9601 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:23.428294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f77120-3ada-42bf-9c79-22afa5d0c4fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=db7cf079fb8e4910afa6e4639616bef9, user_id=d93bf5ee781440c8806d4f77d9d45afc, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:23.432187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f77120-3ada-42bf-9c79-22afa5d0c4fd admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:23.452366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 956/1891] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:23 2022] PUT /identity/v3/projects/db7cf079fb8e4910afa6e4639616bef9/users/d93bf5ee781440c8806d4f77d9d45afc/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:23.459667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:23.461168 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:23.578139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed05c7-19d7-421a-9f9b-f2bad23efa20 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:23.649157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23fa05-5933-487d-9d19-2633ea05ec59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:23.649914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23fa05-5933-487d-9d19-2633ea05ec59 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:23.650124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23fa05-5933-487d-9d19-2633ea05ec59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:23.650398 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23fa05-5933-487d-9d19-2633ea05ec59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:23.651152 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de23fa05-5933-487d-9d19-2633ea05ec59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:23.653511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de23fa05-5933-487d-9d19-2633ea05ec59 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:23.687797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f77120-3ada-42bf-9c79-22afa5d0c4fd admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:23.688218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 936/1892] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:23.703416 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f77120-3ada-42bf-9c79-22afa5d0c4fd admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:23.718785 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c3dc9-61cb-4b13-a5b4-9d11850ab6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:23.720802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c3dc9-61cb-4b13-a5b4-9d11850ab6ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:23.721293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c3dc9-61cb-4b13-a5b4-9d11850ab6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:23.722014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c3dc9-61cb-4b13-a5b4-9d11850ab6ff admin admin] PATH_INFO: `/v3/projects/db7cf079fb8e4910afa6e4639616bef9/users/d93bf5ee781440c8806d4f77d9d45afc/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:23.728378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c3dc9-61cb-4b13-a5b4-9d11850ab6ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=db7cf079fb8e4910afa6e4639616bef9, user_id=d93bf5ee781440c8806d4f77d9d45afc, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:23.734193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c3dc9-61cb-4b13-a5b4-9d11850ab6ff admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:23.746660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:23.771647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 957/1893] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:23 2022] PUT /identity/v3/projects/db7cf079fb8e4910afa6e4639616bef9/users/d93bf5ee781440c8806d4f77d9d45afc/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:23.972022 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23fa05-5933-487d-9d19-2633ea05ec59 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:23.974029 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-de23fa05-5933-487d-9d19-2633ea05ec59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:23.995145 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23fa05-5933-487d-9d19-2633ea05ec59 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:24.107730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:24.109012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:24.109359 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:24.109711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] PATH_INFO: `/v3/projects/c890831defa24509a192fd5aa13fc6a8` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:24.112184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c890831defa24509a192fd5aa13fc6a8)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:24.114890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:24.171988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:24.186788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c890831defa24509a192fd5aa13fc6a8', 'request_id': 'req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:24.205604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c890831defa24509a192fd5aa13fc6a8', 'request_id': 'req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:24.225824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c890831defa24509a192fd5aa13fc6a8', 'request_id': 'req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:24.238974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] The token cache is being invalidate because project c890831defa24509a192fd5aa13fc6a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:24.239403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:24.240295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:24.241995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 937/1894] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:23 2022] DELETE /identity/v3/projects/c890831defa24509a192fd5aa13fc6a8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:24.246022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c3dc9-61cb-4b13-a5b4-9d11850ab6ff admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:24.247705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a75c3dc9-61cb-4b13-a5b4-9d11850ab6ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:24.260999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c3dc9-61cb-4b13-a5b4-9d11850ab6ff admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:24.356783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c56d0-aba0-4710-9aaa-7d38b7e8ce3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:24.357942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c56d0-aba0-4710-9aaa-7d38b7e8ce3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:24.358295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c56d0-aba0-4710-9aaa-7d38b7e8ce3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:24.358781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c56d0-aba0-4710-9aaa-7d38b7e8ce3f admin admin] PATH_INFO: `/v3/users/1a2f2fd5758445eaa02ef0d13ef27240` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:24.360443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c56d0-aba0-4710-9aaa-7d38b7e8ce3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a2f2fd5758445eaa02ef0d13ef27240)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:24.363012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c56d0-aba0-4710-9aaa-7d38b7e8ce3f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:24.413175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb9c56d0-aba0-4710-9aaa-7d38b7e8ce3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a2f2fd5758445eaa02ef0d13ef27240'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:24.424007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb9c56d0-aba0-4710-9aaa-7d38b7e8ce3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a2f2fd5758445eaa02ef0d13ef27240'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:24.424667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb9c56d0-aba0-4710-9aaa-7d38b7e8ce3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a2f2fd5758445eaa02ef0d13ef27240'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:24.425377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb9c56d0-aba0-4710-9aaa-7d38b7e8ce3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a2f2fd5758445eaa02ef0d13ef27240'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:24.452142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb9c56d0-aba0-4710-9aaa-7d38b7e8ce3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a2f2fd5758445eaa02ef0d13ef27240'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:24.473791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 958/1895] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:24 2022] DELETE /identity/v3/users/1a2f2fd5758445eaa02ef0d13ef27240 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:24.882418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:24.898219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be2ee9-2be0-4efe-8fe4-d9a0a082cb24 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:25.046346 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28e744-92b8-42fe-b629-ea89dda850cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:25.047628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28e744-92b8-42fe-b629-ea89dda850cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:25.049324 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28e744-92b8-42fe-b629-ea89dda850cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:25.050025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28e744-92b8-42fe-b629-ea89dda850cf admin admin] PATH_INFO: `/v3/projects/a71c8377119c4b068d68bc138ef15c23` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:25.052746 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28e744-92b8-42fe-b629-ea89dda850cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71c8377119c4b068d68bc138ef15c23)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:25.056317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28e744-92b8-42fe-b629-ea89dda850cf admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:25.165594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5d28e744-92b8-42fe-b629-ea89dda850cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71c8377119c4b068d68bc138ef15c23', 'request_id': 'req-5d28e744-92b8-42fe-b629-ea89dda850cf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:25.187808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5d28e744-92b8-42fe-b629-ea89dda850cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71c8377119c4b068d68bc138ef15c23', 'request_id': 'req-5d28e744-92b8-42fe-b629-ea89dda850cf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:25.239893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5d28e744-92b8-42fe-b629-ea89dda850cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71c8377119c4b068d68bc138ef15c23', 'request_id': 'req-5d28e744-92b8-42fe-b629-ea89dda850cf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:25.282051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5d28e744-92b8-42fe-b629-ea89dda850cf admin admin] The token cache is being invalidate because project a71c8377119c4b068d68bc138ef15c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:25.286231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5d28e744-92b8-42fe-b629-ea89dda850cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:25.290192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5d28e744-92b8-42fe-b629-ea89dda850cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:25.295605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 938/1896] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:24 2022] DELETE /identity/v3/projects/a71c8377119c4b068d68bc138ef15c23 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:25.303154 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a505b-8c5d-442c-931e-2348f089ded8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:25.303824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a505b-8c5d-442c-931e-2348f089ded8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:25.303824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a505b-8c5d-442c-931e-2348f089ded8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:25.359420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ac7a505b-8c5d-442c-931e-2348f089ded8 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:25.360750 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ac7a505b-8c5d-442c-931e-2348f089ded8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:25.541482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 959/1897] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:25.553883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c7f54-3ee2-49ef-ba90-ecea8bd18043 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:25.554394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c7f54-3ee2-49ef-ba90-ecea8bd18043 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:25.554669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c7f54-3ee2-49ef-ba90-ecea8bd18043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:25.599912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b89c7f54-3ee2-49ef-ba90-ecea8bd18043 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:25.601851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b89c7f54-3ee2-49ef-ba90-ecea8bd18043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:25.624608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 939/1898] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:25.629459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a505b-8c5d-442c-931e-2348f089ded8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:25.639561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a505b-8c5d-442c-931e-2348f089ded8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:25.644934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136d8c0-adf4-4e5f-ba04-69bec5d28d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:25.646000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136d8c0-adf4-4e5f-ba04-69bec5d28d89 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:25.646354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136d8c0-adf4-4e5f-ba04-69bec5d28d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:25.646849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136d8c0-adf4-4e5f-ba04-69bec5d28d89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:25.647676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136d8c0-adf4-4e5f-ba04-69bec5d28d89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:25.648291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136d8c0-adf4-4e5f-ba04-69bec5d28d89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:25.650135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136d8c0-adf4-4e5f-ba04-69bec5d28d89 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:25.670362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 960/1899] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:25.676428 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c7f54-3ee2-49ef-ba90-ecea8bd18043 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:25.687077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c7f54-3ee2-49ef-ba90-ecea8bd18043 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:25.692506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca5981-c3ae-4188-845c-fd76a6f72bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:25.693181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca5981-c3ae-4188-845c-fd76a6f72bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:25.693681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca5981-c3ae-4188-845c-fd76a6f72bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:25.693992 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca5981-c3ae-4188-845c-fd76a6f72bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:25.694694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca5981-c3ae-4188-845c-fd76a6f72bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:25.696787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca5981-c3ae-4188-845c-fd76a6f72bf8 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:25.729627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 940/1900] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:25 2022] POST /identity/v3/projects => generated 401 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:25.734314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136d8c0-adf4-4e5f-ba04-69bec5d28d89 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:25.742258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:25.753070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136d8c0-adf4-4e5f-ba04-69bec5d28d89 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:25.796976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f9a3a-68f8-4e8a-8efb-c403212ef462 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:25.799060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f9a3a-68f8-4e8a-8efb-c403212ef462 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:25.799513 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f9a3a-68f8-4e8a-8efb-c403212ef462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:25.833482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-dc2f9a3a-68f8-4e8a-8efb-c403212ef462 None None] MFA Rules not processed for user `d93bf5ee781440c8806d4f77d9d45afc`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:25.835477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dc2f9a3a-68f8-4e8a-8efb-c403212ef462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:25.854795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:25.883629 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597af152-7e94-4abd-8789-20dfc0df7aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:25.884903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597af152-7e94-4abd-8789-20dfc0df7aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:25.885187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597af152-7e94-4abd-8789-20dfc0df7aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:25.885469 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597af152-7e94-4abd-8789-20dfc0df7aa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:25.886223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597af152-7e94-4abd-8789-20dfc0df7aa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:25.888994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597af152-7e94-4abd-8789-20dfc0df7aa2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:25.921020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 961/1901] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:25 2022] POST /identity/v3/users => generated 301 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:25.926370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597af152-7e94-4abd-8789-20dfc0df7aa2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:25.940303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597af152-7e94-4abd-8789-20dfc0df7aa2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:25.947085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3310e-7ae5-4713-96ce-8abc683b579f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:25.947819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3310e-7ae5-4713-96ce-8abc683b579f admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:25.948109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3310e-7ae5-4713-96ce-8abc683b579f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:25.948383 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3310e-7ae5-4713-96ce-8abc683b579f admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:25.949117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3310e-7ae5-4713-96ce-8abc683b579f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:25.952423 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3310e-7ae5-4713-96ce-8abc683b579f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:25.955485 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 941/1902] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:25.967549 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f9a3a-68f8-4e8a-8efb-c403212ef462 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:25.971361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dc2f9a3a-68f8-4e8a-8efb-c403212ef462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:25.994714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 962/1903] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:25.998778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3310e-7ae5-4713-96ce-8abc683b579f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:26.012678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3310e-7ae5-4713-96ce-8abc683b579f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:26.022885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0450a11-08c8-4d9e-8792-898cd671ab67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:26.023653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0450a11-08c8-4d9e-8792-898cd671ab67 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:26.025070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0450a11-08c8-4d9e-8792-898cd671ab67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:26.025441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0450a11-08c8-4d9e-8792-898cd671ab67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:26.026289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0450a11-08c8-4d9e-8792-898cd671ab67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:26.028906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0450a11-08c8-4d9e-8792-898cd671ab67 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:26.051934 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 963/1904] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:26.052900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0450a11-08c8-4d9e-8792-898cd671ab67 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:26.061668 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f9a3a-68f8-4e8a-8efb-c403212ef462 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:26.063550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0450a11-08c8-4d9e-8792-898cd671ab67 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:26.067034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75cbae-358e-4209-9a45-b7fd1d16efd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:26.068348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75cbae-358e-4209-9a45-b7fd1d16efd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:26.069279 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75cbae-358e-4209-9a45-b7fd1d16efd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:26.069774 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9d117-3d84-45b4-826c-9bc29303c6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:26.070246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75cbae-358e-4209-9a45-b7fd1d16efd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:26.070716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9d117-3d84-45b4-826c-9bc29303c6b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:26.071030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9d117-3d84-45b4-826c-9bc29303c6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:26.071340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9d117-3d84-45b4-826c-9bc29303c6b1 admin admin] PATH_INFO: `/v3/projects/f1ea7b382e7f469aab5d2989f167421c/users/5516abd237e642bb956f66fe7c717097/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:26.073867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0d75cbae-358e-4209-9a45-b7fd1d16efd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:26.089567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9d117-3d84-45b4-826c-9bc29303c6b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ea7b382e7f469aab5d2989f167421c, user_id=5516abd237e642bb956f66fe7c717097, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:26.092435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9d117-3d84-45b4-826c-9bc29303c6b1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:26.111536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 964/1905] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:26 2022] PUT /identity/v3/projects/f1ea7b382e7f469aab5d2989f167421c/users/5516abd237e642bb956f66fe7c717097/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:26.122691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27942897-44e7-4388-b830-d78c594a4f26 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:26.122976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27942897-44e7-4388-b830-d78c594a4f26 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:26.123254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27942897-44e7-4388-b830-d78c594a4f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:26.157369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-27942897-44e7-4388-b830-d78c594a4f26 None None] MFA Rules not processed for user `5516abd237e642bb956f66fe7c717097`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:26.159497 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-27942897-44e7-4388-b830-d78c594a4f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:26.165690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75cbae-358e-4209-9a45-b7fd1d16efd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:26.167373 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75cbae-358e-4209-9a45-b7fd1d16efd3 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:26.179693 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 942/1906] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:26.231488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75cbae-358e-4209-9a45-b7fd1d16efd3 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:26.233438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0d75cbae-358e-4209-9a45-b7fd1d16efd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:26.248189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75cbae-358e-4209-9a45-b7fd1d16efd3 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:26.290408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 965/1907] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:26.295275 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27942897-44e7-4388-b830-d78c594a4f26 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:26.351295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4b2a7-d774-43c9-8d63-6cb638153f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:26.352234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4b2a7-d774-43c9-8d63-6cb638153f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:26.352513 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4b2a7-d774-43c9-8d63-6cb638153f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:26.352783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4b2a7-d774-43c9-8d63-6cb638153f5c admin admin] PATH_INFO: `/v3/users/e9057d1866f649df9963f66bcedda2e5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:26.354788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4b2a7-d774-43c9-8d63-6cb638153f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9057d1866f649df9963f66bcedda2e5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:26.356901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4b2a7-d774-43c9-8d63-6cb638153f5c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:26.366669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:26.383560 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:26.414898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27942897-44e7-4388-b830-d78c594a4f26 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:26.421685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df461623-3a6f-4630-b28f-7aa58b860e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:26.422724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df461623-3a6f-4630-b28f-7aa58b860e23 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:26.423767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df461623-3a6f-4630-b28f-7aa58b860e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:26.424681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df461623-3a6f-4630-b28f-7aa58b860e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:26.426228 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df461623-3a6f-4630-b28f-7aa58b860e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:26.429485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df461623-3a6f-4630-b28f-7aa58b860e23 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:26.465260 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e2b4b2a7-d774-43c9-8d63-6cb638153f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9057d1866f649df9963f66bcedda2e5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:26.473253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 966/1908] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:26 2022] POST /identity/v3/projects => generated 399 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:26.478038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df461623-3a6f-4630-b28f-7aa58b860e23 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:26.480573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e2b4b2a7-d774-43c9-8d63-6cb638153f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9057d1866f649df9963f66bcedda2e5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:26.502777 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e2b4b2a7-d774-43c9-8d63-6cb638153f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9057d1866f649df9963f66bcedda2e5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:26.504268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e2b4b2a7-d774-43c9-8d63-6cb638153f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9057d1866f649df9963f66bcedda2e5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:26.518167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e2b4b2a7-d774-43c9-8d63-6cb638153f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9057d1866f649df9963f66bcedda2e5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:26.531513 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 943/1909] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:26 2022] DELETE /identity/v3/users/e9057d1866f649df9963f66bcedda2e5 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:26.536787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e2895-28fa-425f-aa42-f3ea061e7917 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:26.537788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e2895-28fa-425f-aa42-f3ea061e7917 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:26.538221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e2895-28fa-425f-aa42-f3ea061e7917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:26.567154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-884e2895-28fa-425f-aa42-f3ea061e7917 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:26.567819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-884e2895-28fa-425f-aa42-f3ea061e7917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:26.711575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df461623-3a6f-4630-b28f-7aa58b860e23 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:26.775752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 944/1910] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:26.775752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e2895-28fa-425f-aa42-f3ea061e7917 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:26.777843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-884e2895-28fa-425f-aa42-f3ea061e7917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:26.864571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c105b7c-066d-40ee-90ad-ac84c5fd8a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:26.866712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c105b7c-066d-40ee-90ad-ac84c5fd8a33 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:26.867247 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c105b7c-066d-40ee-90ad-ac84c5fd8a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:26.867729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c105b7c-066d-40ee-90ad-ac84c5fd8a33 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:26.868713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c105b7c-066d-40ee-90ad-ac84c5fd8a33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:26.872170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c105b7c-066d-40ee-90ad-ac84c5fd8a33 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:26.905981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e2895-28fa-425f-aa42-f3ea061e7917 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:26.906101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 967/1911] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:26 2022] POST /identity/v3/users => generated 306 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:26.906181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0df83-09a0-45aa-87da-02b6f55bfbc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:26.906791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0df83-09a0-45aa-87da-02b6f55bfbc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:26.907077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0df83-09a0-45aa-87da-02b6f55bfbc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:26.908040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0df83-09a0-45aa-87da-02b6f55bfbc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:26.911123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c105b7c-066d-40ee-90ad-ac84c5fd8a33 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:26.914088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8c105b7c-066d-40ee-90ad-ac84c5fd8a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:26.934696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0df83-09a0-45aa-87da-02b6f55bfbc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:26.936153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0df83-09a0-45aa-87da-02b6f55bfbc2 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:26.946523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c105b7c-066d-40ee-90ad-ac84c5fd8a33 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:26.959910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821d1ec-308c-4d87-b45c-1644b30322e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:26.961759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821d1ec-308c-4d87-b45c-1644b30322e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:26.962056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821d1ec-308c-4d87-b45c-1644b30322e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:26.962336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821d1ec-308c-4d87-b45c-1644b30322e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:26.963078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821d1ec-308c-4d87-b45c-1644b30322e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:26.967819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 945/1912] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:26.968955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821d1ec-308c-4d87-b45c-1644b30322e2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:26.987080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 968/1913] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:26.988214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0df83-09a0-45aa-87da-02b6f55bfbc2 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:27.003622 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0df83-09a0-45aa-87da-02b6f55bfbc2 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:27.015698 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52eb600-7c19-459a-89e8-89110a0c9835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:27.017390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52eb600-7c19-459a-89e8-89110a0c9835 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:27.017808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52eb600-7c19-459a-89e8-89110a0c9835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:27.018179 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52eb600-7c19-459a-89e8-89110a0c9835 admin admin] PATH_INFO: `/v3/projects/1f812956e344459c8816258e25516b31/users/e6bb9245c57342b3aade5054fea6712c/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:27.081039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52eb600-7c19-459a-89e8-89110a0c9835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f812956e344459c8816258e25516b31, user_id=e6bb9245c57342b3aade5054fea6712c, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:27.094181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52eb600-7c19-459a-89e8-89110a0c9835 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:27.137369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 946/1914] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:26 2022] PUT /identity/v3/projects/1f812956e344459c8816258e25516b31/users/e6bb9245c57342b3aade5054fea6712c/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:27.141716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821d1ec-308c-4d87-b45c-1644b30322e2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:27.166844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821d1ec-308c-4d87-b45c-1644b30322e2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:27.260592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:27.329495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8140e9-e4d9-49cd-b536-00a0c776113f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:27.330681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8140e9-e4d9-49cd-b536-00a0c776113f admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:27.331523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8140e9-e4d9-49cd-b536-00a0c776113f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:27.331973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8140e9-e4d9-49cd-b536-00a0c776113f admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:27.332945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8140e9-e4d9-49cd-b536-00a0c776113f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:27.335168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8140e9-e4d9-49cd-b536-00a0c776113f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:27.348722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 969/1915] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:27.352598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52eb600-7c19-459a-89e8-89110a0c9835 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:27.363074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:27.368236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52eb600-7c19-459a-89e8-89110a0c9835 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:27.374572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43925474-1297-4337-8340-44bc155efa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:27.375717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43925474-1297-4337-8340-44bc155efa51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:27.377591 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43925474-1297-4337-8340-44bc155efa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:27.377591 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43925474-1297-4337-8340-44bc155efa51 admin admin] PATH_INFO: `/v3/projects/1f812956e344459c8816258e25516b31/users/e6bb9245c57342b3aade5054fea6712c/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:27.379989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43925474-1297-4337-8340-44bc155efa51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f812956e344459c8816258e25516b31, user_id=e6bb9245c57342b3aade5054fea6712c, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:27.383511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43925474-1297-4337-8340-44bc155efa51 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:27.398616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 947/1916] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:27 2022] PUT /identity/v3/projects/1f812956e344459c8816258e25516b31/users/e6bb9245c57342b3aade5054fea6712c/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:27.410385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72847a1-e7ac-4878-889d-0a8ffa4a0a55 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:27.410824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72847a1-e7ac-4878-889d-0a8ffa4a0a55 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:27.410949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72847a1-e7ac-4878-889d-0a8ffa4a0a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:27.418853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43925474-1297-4337-8340-44bc155efa51 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:27.421131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-43925474-1297-4337-8340-44bc155efa51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:27.453401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b72847a1-e7ac-4878-889d-0a8ffa4a0a55 None None] MFA Rules not processed for user `e6bb9245c57342b3aade5054fea6712c`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:27.455690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b72847a1-e7ac-4878-889d-0a8ffa4a0a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:27.552284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43925474-1297-4337-8340-44bc155efa51 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:27.553503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 970/1917] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:27 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:27.560971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:27.561894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:27.562195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:27.562455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] PATH_INFO: `/v3/projects/193e991a3697426e9663f7e8e136ce05` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:27.564144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] RBAC: Authorizing `identity:delete_project(project_id=193e991a3697426e9663f7e8e136ce05)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:27.564317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72847a1-e7ac-4878-889d-0a8ffa4a0a55 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:27.565722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b72847a1-e7ac-4878-889d-0a8ffa4a0a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:27.566401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:27.619509 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '193e991a3697426e9663f7e8e136ce05', 'request_id': 'req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:27.627636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '193e991a3697426e9663f7e8e136ce05', 'request_id': 'req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:27.634352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '193e991a3697426e9663f7e8e136ce05', 'request_id': 'req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:27.637117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72847a1-e7ac-4878-889d-0a8ffa4a0a55 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:27.645909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] The token cache is being invalidate because project 193e991a3697426e9663f7e8e136ce05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:27.646232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:27.646887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:27.648607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 948/1918] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:27 2022] DELETE /identity/v3/projects/193e991a3697426e9663f7e8e136ce05 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:27.652584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:27.653901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:27.688407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643732c8-e2e6-4334-8bff-27c3dab10730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:27.689596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643732c8-e2e6-4334-8bff-27c3dab10730 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:27.689596 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643732c8-e2e6-4334-8bff-27c3dab10730 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:27.689882 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643732c8-e2e6-4334-8bff-27c3dab10730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:27.691685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-643732c8-e2e6-4334-8bff-27c3dab10730 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:27.719379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a6ead-fb41-45a1-a4c5-1dcb04a3283c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:27.725115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:27.726286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:27.726570 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:27.726845 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] PATH_INFO: `/v3/users/63565d9a97514133ae4ee38043695d3e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:27.728051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63565d9a97514133ae4ee38043695d3e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:27.730080 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:27.733874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:27.752781 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:27.758329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643732c8-e2e6-4334-8bff-27c3dab10730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:27.759824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643732c8-e2e6-4334-8bff-27c3dab10730 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:27.771552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 971/1919] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:27 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:18:27.780063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63565d9a97514133ae4ee38043695d3e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:27.788747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63565d9a97514133ae4ee38043695d3e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:27.802350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63565d9a97514133ae4ee38043695d3e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:27.803056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63565d9a97514133ae4ee38043695d3e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:27.812253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63565d9a97514133ae4ee38043695d3e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:27.819866 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 949/1920] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:27 2022] DELETE /identity/v3/users/63565d9a97514133ae4ee38043695d3e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:27.851159 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643732c8-e2e6-4334-8bff-27c3dab10730 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:27.852561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-643732c8-e2e6-4334-8bff-27c3dab10730 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:27.863207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643732c8-e2e6-4334-8bff-27c3dab10730 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:27.915936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca49122-6407-4ae7-a0e2-1b986822738b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:27.916664 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca49122-6407-4ae7-a0e2-1b986822738b service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:27.916971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca49122-6407-4ae7-a0e2-1b986822738b service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:27.917273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca49122-6407-4ae7-a0e2-1b986822738b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:27.919773 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7ca49122-6407-4ae7-a0e2-1b986822738b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:27.984713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca49122-6407-4ae7-a0e2-1b986822738b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:27.986292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca49122-6407-4ae7-a0e2-1b986822738b service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:27.995820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 972/1921] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:28.103573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:28.105240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:28.220826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd2b20-0093-4b7e-8a77-4c473b585d17 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:28.227795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:28.229016 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:28.229615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:28.230154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] PATH_INFO: `/v3/projects/7620cf483bfc45658d659e77d41d28ff` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:28.231643 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7620cf483bfc45658d659e77d41d28ff)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:28.233628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:28.264527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca49122-6407-4ae7-a0e2-1b986822738b service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:28.273700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7ca49122-6407-4ae7-a0e2-1b986822738b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:28.283290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca49122-6407-4ae7-a0e2-1b986822738b service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:28.292461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef65802-4fa7-4677-a489-fadfe130abbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:28.293669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef65802-4fa7-4677-a489-fadfe130abbe service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:28.294632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef65802-4fa7-4677-a489-fadfe130abbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:28.294975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef65802-4fa7-4677-a489-fadfe130abbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:28.298639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5ef65802-4fa7-4677-a489-fadfe130abbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:28.347405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7620cf483bfc45658d659e77d41d28ff', 'request_id': 'req-eee07781-7159-4617-963e-ba15a6412215'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:28.366104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7620cf483bfc45658d659e77d41d28ff', 'request_id': 'req-eee07781-7159-4617-963e-ba15a6412215'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:28.373863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7620cf483bfc45658d659e77d41d28ff', 'request_id': 'req-eee07781-7159-4617-963e-ba15a6412215'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:28.400557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] The token cache is being invalidate because project 7620cf483bfc45658d659e77d41d28ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:28.400847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:28.401523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:28.403433 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 950/1922] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:28 2022] DELETE /identity/v3/projects/7620cf483bfc45658d659e77d41d28ff => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:28.406984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:28.410732 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:28.461302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef65802-4fa7-4677-a489-fadfe130abbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:28.463755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef65802-4fa7-4677-a489-fadfe130abbe service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:28.479934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:28.483496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 973/1923] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:28.530968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee07781-7159-4617-963e-ba15a6412215 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:28.538032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962c115-1c4b-4fd8-8e07-915bb06a9fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:28.539190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962c115-1c4b-4fd8-8e07-915bb06a9fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:28.539536 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962c115-1c4b-4fd8-8e07-915bb06a9fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:28.539873 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962c115-1c4b-4fd8-8e07-915bb06a9fb6 admin admin] PATH_INFO: `/v3/users/7a0f3312bb2c4b2789e4cd6755e820a9` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:28.541175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962c115-1c4b-4fd8-8e07-915bb06a9fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a0f3312bb2c4b2789e4cd6755e820a9)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:28.543200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962c115-1c4b-4fd8-8e07-915bb06a9fb6 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:28.632590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5962c115-1c4b-4fd8-8e07-915bb06a9fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a0f3312bb2c4b2789e4cd6755e820a9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:28.652932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5962c115-1c4b-4fd8-8e07-915bb06a9fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a0f3312bb2c4b2789e4cd6755e820a9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:28.679822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5962c115-1c4b-4fd8-8e07-915bb06a9fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a0f3312bb2c4b2789e4cd6755e820a9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:28.680723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5962c115-1c4b-4fd8-8e07-915bb06a9fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a0f3312bb2c4b2789e4cd6755e820a9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:28.694541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5962c115-1c4b-4fd8-8e07-915bb06a9fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a0f3312bb2c4b2789e4cd6755e820a9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:28.702635 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 951/1924] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:28 2022] DELETE /identity/v3/users/7a0f3312bb2c4b2789e4cd6755e820a9 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:29.000949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef65802-4fa7-4677-a489-fadfe130abbe service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:29.012344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef65802-4fa7-4677-a489-fadfe130abbe service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:29.031893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:29.071984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:29.072777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:29.073099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:29.073378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b admin admin] PATH_INFO: `/v3/projects/966fa72eb0634a4f95f16fb9a96edc9e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:29.074631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=966fa72eb0634a4f95f16fb9a96edc9e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:29.076558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:29.132062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966fa72eb0634a4f95f16fb9a96edc9e', 'request_id': 'req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:29.138182 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966fa72eb0634a4f95f16fb9a96edc9e', 'request_id': 'req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:29.143745 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966fa72eb0634a4f95f16fb9a96edc9e', 'request_id': 'req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:29.156255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b admin admin] The token cache is being invalidate because project 966fa72eb0634a4f95f16fb9a96edc9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:29.156547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:29.157187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7f56c1b6-1417-4782-b66f-4f7de2b46d8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:29.158886 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 974/1925] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:28 2022] DELETE /identity/v3/projects/966fa72eb0634a4f95f16fb9a96edc9e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:29.163779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01daff9-4eb7-4cc9-81a6-d6dd54b293ec None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:29.164070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01daff9-4eb7-4cc9-81a6-d6dd54b293ec None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:29.164346 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01daff9-4eb7-4cc9-81a6-d6dd54b293ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:29.183142 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e01daff9-4eb7-4cc9-81a6-d6dd54b293ec None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:29.184342 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e01daff9-4eb7-4cc9-81a6-d6dd54b293ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:29.258668 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 952/1926] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:29.266719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c5e15-b8e4-4196-8667-4a220f920446 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:29.267287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c5e15-b8e4-4196-8667-4a220f920446 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:29.267603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c5e15-b8e4-4196-8667-4a220f920446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:29.298570 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-944c5e15-b8e4-4196-8667-4a220f920446 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:29.299990 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-944c5e15-b8e4-4196-8667-4a220f920446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:29.317827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 975/1927] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:29.320133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01daff9-4eb7-4cc9-81a6-d6dd54b293ec None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:29.337539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01daff9-4eb7-4cc9-81a6-d6dd54b293ec None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:29.343712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a7f35-c23e-450a-9096-8d4242a0cfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:29.344968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7f35-c23e-450a-9096-8d4242a0cfe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:29.345286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7f35-c23e-450a-9096-8d4242a0cfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:29.345648 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7f35-c23e-450a-9096-8d4242a0cfe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:29.346605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a7f35-c23e-450a-9096-8d4242a0cfe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:29.346732 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a7f35-c23e-450a-9096-8d4242a0cfe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:29.348339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a7f35-c23e-450a-9096-8d4242a0cfe0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:29.362286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 953/1928] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:29.367152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c5e15-b8e4-4196-8667-4a220f920446 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:29.379460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c5e15-b8e4-4196-8667-4a220f920446 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:29.386865 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee15e98-ee0d-4279-865f-3dafca11d6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:29.387564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee15e98-ee0d-4279-865f-3dafca11d6c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:29.387843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee15e98-ee0d-4279-865f-3dafca11d6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:29.388116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee15e98-ee0d-4279-865f-3dafca11d6c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:29.388823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee15e98-ee0d-4279-865f-3dafca11d6c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:29.390937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee15e98-ee0d-4279-865f-3dafca11d6c5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:29.433973 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 976/1929] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:29 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:29.438242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a7f35-c23e-450a-9096-8d4242a0cfe0 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:29.455404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a7f35-c23e-450a-9096-8d4242a0cfe0 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:29.500492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:29.549144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c4b6d-9c36-4cdd-bfa8-27c6e5700b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:29.550743 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c4b6d-9c36-4cdd-bfa8-27c6e5700b12 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:29.551095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c4b6d-9c36-4cdd-bfa8-27c6e5700b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:29.551410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c4b6d-9c36-4cdd-bfa8-27c6e5700b12 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:29.552096 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c4b6d-9c36-4cdd-bfa8-27c6e5700b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:29.554628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c4b6d-9c36-4cdd-bfa8-27c6e5700b12 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:29.594501 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 954/1930] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:29 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:29.596738 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee15e98-ee0d-4279-865f-3dafca11d6c5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:29.633085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee15e98-ee0d-4279-865f-3dafca11d6c5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:29.641216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7566c-847e-4dde-bc80-faaebf0bdc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:29.642960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7566c-847e-4dde-bc80-faaebf0bdc1e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:29.643377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7566c-847e-4dde-bc80-faaebf0bdc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:29.643812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7566c-847e-4dde-bc80-faaebf0bdc1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:29.644741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7566c-847e-4dde-bc80-faaebf0bdc1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:29.647507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7566c-847e-4dde-bc80-faaebf0bdc1e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:29.668196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 977/1931] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:29.670163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c4b6d-9c36-4cdd-bfa8-27c6e5700b12 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:29.689377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c4b6d-9c36-4cdd-bfa8-27c6e5700b12 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:29.703008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51aae5-dda0-4be4-8bc9-11764e67616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:29.705989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51aae5-dda0-4be4-8bc9-11764e67616e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:29.706333 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51aae5-dda0-4be4-8bc9-11764e67616e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:29.706749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51aae5-dda0-4be4-8bc9-11764e67616e admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:29.718618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51aae5-dda0-4be4-8bc9-11764e67616e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:29.718618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51aae5-dda0-4be4-8bc9-11764e67616e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:29.745236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7566c-847e-4dde-bc80-faaebf0bdc1e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:29.748327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 955/1932] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:29.770380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7566c-847e-4dde-bc80-faaebf0bdc1e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:29.781225 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cd666-6225-4c64-9f35-d912cc176514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:29.783102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cd666-6225-4c64-9f35-d912cc176514 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:29.783532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cd666-6225-4c64-9f35-d912cc176514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:29.784051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cd666-6225-4c64-9f35-d912cc176514 admin admin] PATH_INFO: `/v3/projects/0f4b3a0033364551bed840f854ce09db/users/b3885acd272a499ca8d54acc0548e8a7/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:29.832072 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19cd666-6225-4c64-9f35-d912cc176514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4b3a0033364551bed840f854ce09db, user_id=b3885acd272a499ca8d54acc0548e8a7, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:29.833719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19cd666-6225-4c64-9f35-d912cc176514 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:29.865158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 978/1933] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:29 2022] PUT /identity/v3/projects/0f4b3a0033364551bed840f854ce09db/users/b3885acd272a499ca8d54acc0548e8a7/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:29.878920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b184f-e221-46b1-a23b-d7760945b4a3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:29.879474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b184f-e221-46b1-a23b-d7760945b4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:29.880357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b184f-e221-46b1-a23b-d7760945b4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:29.919540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-406b184f-e221-46b1-a23b-d7760945b4a3 None None] MFA Rules not processed for user `b3885acd272a499ca8d54acc0548e8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:29.921494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-406b184f-e221-46b1-a23b-d7760945b4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:30.053351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 956/1934] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:30.058189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cd666-6225-4c64-9f35-d912cc176514 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:30.181645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:30.193426 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cd666-6225-4c64-9f35-d912cc176514 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:30.203508 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f7515-fff2-41d4-8342-8d1ae5c47da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:30.205701 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f7515-fff2-41d4-8342-8d1ae5c47da0 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:30.207235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f7515-fff2-41d4-8342-8d1ae5c47da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:30.207585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f7515-fff2-41d4-8342-8d1ae5c47da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:30.208913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f7515-fff2-41d4-8342-8d1ae5c47da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:30.212965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f7515-fff2-41d4-8342-8d1ae5c47da0 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:30.259167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 979/1935] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:30 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:30.263406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b184f-e221-46b1-a23b-d7760945b4a3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:30.278119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b184f-e221-46b1-a23b-d7760945b4a3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:30.411917 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9989d4-7026-4d16-b651-7c3e4041b24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:30.415954 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9989d4-7026-4d16-b651-7c3e4041b24c admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:30.416255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9989d4-7026-4d16-b651-7c3e4041b24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:30.416735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9989d4-7026-4d16-b651-7c3e4041b24c admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:30.429239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9989d4-7026-4d16-b651-7c3e4041b24c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:30.435079 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9989d4-7026-4d16-b651-7c3e4041b24c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:30.464964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 957/1936] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:30 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:30.469824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f7515-fff2-41d4-8342-8d1ae5c47da0 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:30.483383 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f7515-fff2-41d4-8342-8d1ae5c47da0 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:30.492385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f0857-ed92-4a13-9447-ba45965fe058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:30.493826 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f0857-ed92-4a13-9447-ba45965fe058 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:30.494470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f0857-ed92-4a13-9447-ba45965fe058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:30.494945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f0857-ed92-4a13-9447-ba45965fe058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:30.496169 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f0857-ed92-4a13-9447-ba45965fe058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:30.498519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f0857-ed92-4a13-9447-ba45965fe058 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:30.522881 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 980/1937] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:30.523067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9989d4-7026-4d16-b651-7c3e4041b24c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:30.539748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9989d4-7026-4d16-b651-7c3e4041b24c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:30.545502 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83bde1-0c13-4ead-9a2c-4f6c41f73e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:30.546615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83bde1-0c13-4ead-9a2c-4f6c41f73e86 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:30.546898 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83bde1-0c13-4ead-9a2c-4f6c41f73e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:30.547178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83bde1-0c13-4ead-9a2c-4f6c41f73e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:30.548809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83bde1-0c13-4ead-9a2c-4f6c41f73e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:30.550598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83bde1-0c13-4ead-9a2c-4f6c41f73e86 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:30.571129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 958/1938] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:30.575319 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f0857-ed92-4a13-9447-ba45965fe058 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:30.591778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f0857-ed92-4a13-9447-ba45965fe058 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:30.604448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04c8ae-a1cc-422e-8556-8d469b28aa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:30.607984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04c8ae-a1cc-422e-8556-8d469b28aa15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:30.608399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04c8ae-a1cc-422e-8556-8d469b28aa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:30.608803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04c8ae-a1cc-422e-8556-8d469b28aa15 admin admin] PATH_INFO: `/v3/projects/5b187b7c80db4ad5a9b17044aaf381ba/users/23ea695e276e44cfb1d483c21aefb71c/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:30.635585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04c8ae-a1cc-422e-8556-8d469b28aa15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b187b7c80db4ad5a9b17044aaf381ba, user_id=23ea695e276e44cfb1d483c21aefb71c, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:30.638950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04c8ae-a1cc-422e-8556-8d469b28aa15 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:30.653834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 981/1939] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:30 2022] PUT /identity/v3/projects/5b187b7c80db4ad5a9b17044aaf381ba/users/23ea695e276e44cfb1d483c21aefb71c/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:30.664924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc61d2-17fe-46be-b3e4-65bc1750b7ab None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:30.665210 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc61d2-17fe-46be-b3e4-65bc1750b7ab None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:30.665485 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc61d2-17fe-46be-b3e4-65bc1750b7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:30.697366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b7cc61d2-17fe-46be-b3e4-65bc1750b7ab None None] MFA Rules not processed for user `23ea695e276e44cfb1d483c21aefb71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:30.698645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b7cc61d2-17fe-46be-b3e4-65bc1750b7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:30.711071 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:30.771533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 959/1940] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:30.775760 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04c8ae-a1cc-422e-8556-8d469b28aa15 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:30.859673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04c8ae-a1cc-422e-8556-8d469b28aa15 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:30.865135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d2dc0-7eb5-4e58-8b37-ab6e4a4f0cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:30.865913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d2dc0-7eb5-4e58-8b37-ab6e4a4f0cce admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:30.866266 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d2dc0-7eb5-4e58-8b37-ab6e4a4f0cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:30.866714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d2dc0-7eb5-4e58-8b37-ab6e4a4f0cce admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:30.867603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d2dc0-7eb5-4e58-8b37-ab6e4a4f0cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:30.869759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d2dc0-7eb5-4e58-8b37-ab6e4a4f0cce admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:30.910848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 982/1941] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:30 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:30.912758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc61d2-17fe-46be-b3e4-65bc1750b7ab None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:30.928022 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc61d2-17fe-46be-b3e4-65bc1750b7ab None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:30.993608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040b053-b9a4-4f0c-8c43-89186110a355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:30.994386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040b053-b9a4-4f0c-8c43-89186110a355 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:30.994668 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040b053-b9a4-4f0c-8c43-89186110a355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:30.994945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040b053-b9a4-4f0c-8c43-89186110a355 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:30.995710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040b053-b9a4-4f0c-8c43-89186110a355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:31.008101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040b053-b9a4-4f0c-8c43-89186110a355 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:31.027284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 960/1942] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:30 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:31.031628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d2dc0-7eb5-4e58-8b37-ab6e4a4f0cce admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:31.043046 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d2dc0-7eb5-4e58-8b37-ab6e4a4f0cce admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:31.048251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527b9b4-a52b-4f83-8771-9a4696790a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:31.049984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527b9b4-a52b-4f83-8771-9a4696790a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:31.050270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527b9b4-a52b-4f83-8771-9a4696790a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:31.050550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527b9b4-a52b-4f83-8771-9a4696790a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:31.051279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527b9b4-a52b-4f83-8771-9a4696790a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:31.052942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527b9b4-a52b-4f83-8771-9a4696790a62 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:31.065297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 983/1943] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:31.074971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040b053-b9a4-4f0c-8c43-89186110a355 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:31.088635 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040b053-b9a4-4f0c-8c43-89186110a355 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:31.094091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a4ffe-8820-48b3-ae11-92111b84768c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:31.095045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a4ffe-8820-48b3-ae11-92111b84768c admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:31.096121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a4ffe-8820-48b3-ae11-92111b84768c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:31.096121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a4ffe-8820-48b3-ae11-92111b84768c admin admin] PATH_INFO: `/v3/projects/c59d02acefb44b9b8651069e9121e02e/users/112d4a2102ca437587b753965590420d/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:31.119122 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a4ffe-8820-48b3-ae11-92111b84768c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c59d02acefb44b9b8651069e9121e02e, user_id=112d4a2102ca437587b753965590420d, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:31.123610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a4ffe-8820-48b3-ae11-92111b84768c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:31.140473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 961/1944] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:31 2022] PUT /identity/v3/projects/c59d02acefb44b9b8651069e9121e02e/users/112d4a2102ca437587b753965590420d/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:31.145334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527b9b4-a52b-4f83-8771-9a4696790a62 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:31.155424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527b9b4-a52b-4f83-8771-9a4696790a62 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:31.220159 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df9c03-eb55-4d55-84b5-cf4537b21577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:31.221274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df9c03-eb55-4d55-84b5-cf4537b21577 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:31.221827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df9c03-eb55-4d55-84b5-cf4537b21577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:31.222251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df9c03-eb55-4d55-84b5-cf4537b21577 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:31.223217 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df9c03-eb55-4d55-84b5-cf4537b21577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:31.224986 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df9c03-eb55-4d55-84b5-cf4537b21577 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:31.240472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 984/1945] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:31.241463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a4ffe-8820-48b3-ae11-92111b84768c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:31.254311 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a4ffe-8820-48b3-ae11-92111b84768c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:31.259039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c9dd9-97a7-4fc2-9a5f-9e07f9e0f8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:31.260040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c9dd9-97a7-4fc2-9a5f-9e07f9e0f8b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:31.260424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c9dd9-97a7-4fc2-9a5f-9e07f9e0f8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:31.260804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c9dd9-97a7-4fc2-9a5f-9e07f9e0f8b7 admin admin] PATH_INFO: `/v3/projects/c59d02acefb44b9b8651069e9121e02e/users/112d4a2102ca437587b753965590420d/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:31.262960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c9dd9-97a7-4fc2-9a5f-9e07f9e0f8b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c59d02acefb44b9b8651069e9121e02e, user_id=112d4a2102ca437587b753965590420d, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:31.265718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c9dd9-97a7-4fc2-9a5f-9e07f9e0f8b7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:31.286386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 962/1946] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:31 2022] PUT /identity/v3/projects/c59d02acefb44b9b8651069e9121e02e/users/112d4a2102ca437587b753965590420d/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:31.298575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe00c3e-debe-4882-bd05-90911c51bbdc None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:31.298894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe00c3e-debe-4882-bd05-90911c51bbdc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:31.299218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe00c3e-debe-4882-bd05-90911c51bbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:31.318333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:31.330397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-0fe00c3e-debe-4882-bd05-90911c51bbdc None None] MFA Rules not processed for user `112d4a2102ca437587b753965590420d`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:31.331648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0fe00c3e-debe-4882-bd05-90911c51bbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:31.411661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 985/1947] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:31.419104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c9dd9-97a7-4fc2-9a5f-9e07f9e0f8b7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:31.421440 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a34c9dd9-97a7-4fc2-9a5f-9e07f9e0f8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:31.487102 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c9dd9-97a7-4fc2-9a5f-9e07f9e0f8b7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:31.490910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab103bae-e4a1-4124-8b41-122094ede5d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:31.491604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab103bae-e4a1-4124-8b41-122094ede5d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:31.491888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab103bae-e4a1-4124-8b41-122094ede5d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:31.492160 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab103bae-e4a1-4124-8b41-122094ede5d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:31.517023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:31.558871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab103bae-e4a1-4124-8b41-122094ede5d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:31.560394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab103bae-e4a1-4124-8b41-122094ede5d3 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:31.571376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 963/1948] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:38.586064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe00c3e-debe-4882-bd05-90911c51bbdc None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:38.588143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0fe00c3e-debe-4882-bd05-90911c51bbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:38.604378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe00c3e-debe-4882-bd05-90911c51bbdc None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:38.608479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0955ef8-5979-4dcd-a487-e174803937f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:38.609228 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0955ef8-5979-4dcd-a487-e174803937f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:38.609634 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0955ef8-5979-4dcd-a487-e174803937f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:38.610378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0955ef8-5979-4dcd-a487-e174803937f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:38.706221 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0955ef8-5979-4dcd-a487-e174803937f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:38.707769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0955ef8-5979-4dcd-a487-e174803937f7 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:38.718310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab103bae-e4a1-4124-8b41-122094ede5d3 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:38.720327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ab103bae-e4a1-4124-8b41-122094ede5d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:38.733828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 986/1949] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:38.805332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab103bae-e4a1-4124-8b41-122094ede5d3 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:38.810792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:38.811770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:38.812125 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:38.812407 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] PATH_INFO: `/v3/users/5516abd237e642bb956f66fe7c717097` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:38.813610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5516abd237e642bb956f66fe7c717097)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:38.815679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:38.871766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5516abd237e642bb956f66fe7c717097'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:38.883080 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5516abd237e642bb956f66fe7c717097'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:38.900976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5516abd237e642bb956f66fe7c717097'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:38.901825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5516abd237e642bb956f66fe7c717097'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:38.921633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5516abd237e642bb956f66fe7c717097'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:38.936159 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 964/1950] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:38 2022] DELETE /identity/v3/users/5516abd237e642bb956f66fe7c717097 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:38.942161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11219c-2dfe-4905-b4c9-434a858fe1ad None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:38.942449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11219c-2dfe-4905-b4c9-434a858fe1ad None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:38.942632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11219c-2dfe-4905-b4c9-434a858fe1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:38.962242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8e11219c-2dfe-4905-b4c9-434a858fe1ad None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:38.963954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8e11219c-2dfe-4905-b4c9-434a858fe1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:39.106624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 987/1951] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:39.114532 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:39.116491 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:39.166340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:39.187326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa8373-3eff-46ae-9886-bb428e794389 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:39.192356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae47bf-4b1f-43af-bce0-24c03b48a396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:39.193493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae47bf-4b1f-43af-bce0-24c03b48a396 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:39.195382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae47bf-4b1f-43af-bce0-24c03b48a396 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:39.195382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae47bf-4b1f-43af-bce0-24c03b48a396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:39.212725 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae47bf-4b1f-43af-bce0-24c03b48a396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:39.214371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae47bf-4b1f-43af-bce0-24c03b48a396 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:39.228246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 965/1952] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:18:39.482577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11219c-2dfe-4905-b4c9-434a858fe1ad None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:39.484041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8e11219c-2dfe-4905-b4c9-434a858fe1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:39.539852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:39.623350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11219c-2dfe-4905-b4c9-434a858fe1ad None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:39.628622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba36f0a-7381-4121-b819-2195f39c1d17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:39.629295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba36f0a-7381-4121-b819-2195f39c1d17 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:39.629582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba36f0a-7381-4121-b819-2195f39c1d17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:39.629894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba36f0a-7381-4121-b819-2195f39c1d17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:39.632817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4ba36f0a-7381-4121-b819-2195f39c1d17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:39.721434 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae47bf-4b1f-43af-bce0-24c03b48a396 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:39.723091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3bae47bf-4b1f-43af-bce0-24c03b48a396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:39.742840 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae47bf-4b1f-43af-bce0-24c03b48a396 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:39.760547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:39.761424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:39.761708 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:39.762010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] PATH_INFO: `/v3/projects/f1ea7b382e7f469aab5d2989f167421c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:39.762411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba36f0a-7381-4121-b819-2195f39c1d17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:39.764623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba36f0a-7381-4121-b819-2195f39c1d17 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:39.764744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1ea7b382e7f469aab5d2989f167421c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:39.767048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:39.793850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 988/1953] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:39.856548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1ea7b382e7f469aab5d2989f167421c', 'request_id': 'req-88d2e668-acab-4dbf-aa10-514fb9d3bc73'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.866116 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1ea7b382e7f469aab5d2989f167421c', 'request_id': 'req-88d2e668-acab-4dbf-aa10-514fb9d3bc73'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.876564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1ea7b382e7f469aab5d2989f167421c', 'request_id': 'req-88d2e668-acab-4dbf-aa10-514fb9d3bc73'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.891729 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] The token cache is being invalidate because project f1ea7b382e7f469aab5d2989f167421c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:39.892308 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.893387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.895652 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 966/1954] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:39 2022] DELETE /identity/v3/projects/f1ea7b382e7f469aab5d2989f167421c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:39.899653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba36f0a-7381-4121-b819-2195f39c1d17 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:39.914928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba36f0a-7381-4121-b819-2195f39c1d17 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:39.981083 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabe88a-1cc7-4345-b5e2-5d2f7b0e3ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:39.982109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabe88a-1cc7-4345-b5e2-5d2f7b0e3ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:39.982464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabe88a-1cc7-4345-b5e2-5d2f7b0e3ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:39.982800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabe88a-1cc7-4345-b5e2-5d2f7b0e3ca9 admin admin] PATH_INFO: `/v3/users/e6bb9245c57342b3aade5054fea6712c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:39.984678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabe88a-1cc7-4345-b5e2-5d2f7b0e3ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6bb9245c57342b3aade5054fea6712c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:39.986936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabe88a-1cc7-4345-b5e2-5d2f7b0e3ca9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:40.037848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7eabe88a-1cc7-4345-b5e2-5d2f7b0e3ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6bb9245c57342b3aade5054fea6712c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.046399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7eabe88a-1cc7-4345-b5e2-5d2f7b0e3ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6bb9245c57342b3aade5054fea6712c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.047296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7eabe88a-1cc7-4345-b5e2-5d2f7b0e3ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6bb9245c57342b3aade5054fea6712c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.047953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7eabe88a-1cc7-4345-b5e2-5d2f7b0e3ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6bb9245c57342b3aade5054fea6712c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.062593 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7eabe88a-1cc7-4345-b5e2-5d2f7b0e3ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6bb9245c57342b3aade5054fea6712c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.082353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 989/1955] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:39 2022] DELETE /identity/v3/users/e6bb9245c57342b3aade5054fea6712c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:40.381183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:40.382681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:40.398639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2e668-acab-4dbf-aa10-514fb9d3bc73 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:40.459160 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0bb83-e773-4383-9dd7-61bce83d3df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:40.460040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0bb83-e773-4383-9dd7-61bce83d3df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:40.460318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0bb83-e773-4383-9dd7-61bce83d3df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:40.460595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0bb83-e773-4383-9dd7-61bce83d3df7 admin admin] PATH_INFO: `/v3/projects/1f812956e344459c8816258e25516b31` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:40.461790 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0bb83-e773-4383-9dd7-61bce83d3df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f812956e344459c8816258e25516b31)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:40.463699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0bb83-e773-4383-9dd7-61bce83d3df7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:40.519317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d4b0bb83-e773-4383-9dd7-61bce83d3df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f812956e344459c8816258e25516b31', 'request_id': 'req-d4b0bb83-e773-4383-9dd7-61bce83d3df7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.528226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d4b0bb83-e773-4383-9dd7-61bce83d3df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f812956e344459c8816258e25516b31', 'request_id': 'req-d4b0bb83-e773-4383-9dd7-61bce83d3df7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.534947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d4b0bb83-e773-4383-9dd7-61bce83d3df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f812956e344459c8816258e25516b31', 'request_id': 'req-d4b0bb83-e773-4383-9dd7-61bce83d3df7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.540934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:40.548268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d4b0bb83-e773-4383-9dd7-61bce83d3df7 admin admin] The token cache is being invalidate because project 1f812956e344459c8816258e25516b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:40.549318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d4b0bb83-e773-4383-9dd7-61bce83d3df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.550333 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d4b0bb83-e773-4383-9dd7-61bce83d3df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.552330 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 967/1956] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:40 2022] DELETE /identity/v3/projects/1f812956e344459c8816258e25516b31 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:40.557196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a518fd-7b36-45e5-ba78-214c2bd773f0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:40.557487 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a518fd-7b36-45e5-ba78-214c2bd773f0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:40.557798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a518fd-7b36-45e5-ba78-214c2bd773f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:40.565051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:40.579454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-37a518fd-7b36-45e5-ba78-214c2bd773f0 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:40.579628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-37a518fd-7b36-45e5-ba78-214c2bd773f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:40.675105 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 990/1957] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:40.678223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796e4f5-683e-43ce-a01a-6cf2d70236c0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:40.678515 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796e4f5-683e-43ce-a01a-6cf2d70236c0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:40.678787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796e4f5-683e-43ce-a01a-6cf2d70236c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:40.712756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7796e4f5-683e-43ce-a01a-6cf2d70236c0 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:40.714454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7796e4f5-683e-43ce-a01a-6cf2d70236c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:40.728150 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 968/1958] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:40.729503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a518fd-7b36-45e5-ba78-214c2bd773f0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:40.749412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a518fd-7b36-45e5-ba78-214c2bd773f0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:40.756310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b4059-08b7-4481-b40b-89c505899713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:40.757067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b4059-08b7-4481-b40b-89c505899713 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:40.757353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b4059-08b7-4481-b40b-89c505899713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:40.757628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b4059-08b7-4481-b40b-89c505899713 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:40.758408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b4059-08b7-4481-b40b-89c505899713 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:40.758865 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b4059-08b7-4481-b40b-89c505899713 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:40.760796 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b4059-08b7-4481-b40b-89c505899713 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:40.778642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 991/1959] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:40.780095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796e4f5-683e-43ce-a01a-6cf2d70236c0 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:40.790441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796e4f5-683e-43ce-a01a-6cf2d70236c0 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:40.796364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609e854-debd-4930-aefc-8a5b2d3f21d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:40.797383 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609e854-debd-4930-aefc-8a5b2d3f21d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:40.797780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609e854-debd-4930-aefc-8a5b2d3f21d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:40.798061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609e854-debd-4930-aefc-8a5b2d3f21d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:40.800752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609e854-debd-4930-aefc-8a5b2d3f21d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:40.803043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609e854-debd-4930-aefc-8a5b2d3f21d2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:40.831774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 969/1960] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:40 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:40.836062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b4059-08b7-4481-b40b-89c505899713 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:40.846769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b4059-08b7-4481-b40b-89c505899713 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:40.908257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd5e48-60e9-48c8-94d6-e303cb1b8f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:40.909149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd5e48-60e9-48c8-94d6-e303cb1b8f05 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:40.909432 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd5e48-60e9-48c8-94d6-e303cb1b8f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:40.909701 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd5e48-60e9-48c8-94d6-e303cb1b8f05 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:40.910424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd5e48-60e9-48c8-94d6-e303cb1b8f05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:40.912350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd5e48-60e9-48c8-94d6-e303cb1b8f05 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:40.945144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 992/1961] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:40 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:40.946895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609e854-debd-4930-aefc-8a5b2d3f21d2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:40.960812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609e854-debd-4930-aefc-8a5b2d3f21d2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:40.968888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c11a7-0275-44af-b5cb-252165a4ecc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:40.969586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c11a7-0275-44af-b5cb-252165a4ecc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:40.969919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c11a7-0275-44af-b5cb-252165a4ecc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:40.970199 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c11a7-0275-44af-b5cb-252165a4ecc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:40.971145 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c11a7-0275-44af-b5cb-252165a4ecc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:40.973396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c11a7-0275-44af-b5cb-252165a4ecc4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:40.992923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd5e48-60e9-48c8-94d6-e303cb1b8f05 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.005759 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 970/1962] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:41.008354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd5e48-60e9-48c8-94d6-e303cb1b8f05 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.015884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70ab0b-72f9-469a-a45d-633727b33fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.015884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70ab0b-72f9-469a-a45d-633727b33fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.015884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70ab0b-72f9-469a-a45d-633727b33fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.015884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70ab0b-72f9-469a-a45d-633727b33fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.026868 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70ab0b-72f9-469a-a45d-633727b33fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.031601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70ab0b-72f9-469a-a45d-633727b33fd1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.065007 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c11a7-0275-44af-b5cb-252165a4ecc4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.065490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 993/1963] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:41.086356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c11a7-0275-44af-b5cb-252165a4ecc4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.092484 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77129e12-7c6b-42ca-aecf-496566d77bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.093609 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77129e12-7c6b-42ca-aecf-496566d77bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.093982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77129e12-7c6b-42ca-aecf-496566d77bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.094314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77129e12-7c6b-42ca-aecf-496566d77bbe admin admin] PATH_INFO: `/v3/projects/f735a218aee64ec1a8bd43a69b566335/users/5309a00c37d94746b24f35882f53ebdf/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.122662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77129e12-7c6b-42ca-aecf-496566d77bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f735a218aee64ec1a8bd43a69b566335, user_id=5309a00c37d94746b24f35882f53ebdf, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.125547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77129e12-7c6b-42ca-aecf-496566d77bbe admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.159923 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 971/1964] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:41 2022] PUT /identity/v3/projects/f735a218aee64ec1a8bd43a69b566335/users/5309a00c37d94746b24f35882f53ebdf/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:41.166555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b29bc-e642-4b5f-9b32-40bae0cfc3bb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.166848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b29bc-e642-4b5f-9b32-40bae0cfc3bb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.167129 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b29bc-e642-4b5f-9b32-40bae0cfc3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.210399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a70b29bc-e642-4b5f-9b32-40bae0cfc3bb None None] MFA Rules not processed for user `5309a00c37d94746b24f35882f53ebdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:41.211915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a70b29bc-e642-4b5f-9b32-40bae0cfc3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:41.335395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:41.340779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 994/1965] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:41.347558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77129e12-7c6b-42ca-aecf-496566d77bbe admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.478916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77129e12-7c6b-42ca-aecf-496566d77bbe admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.489625 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271134d-2881-49d6-b6ca-4d02dd2db562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.492244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271134d-2881-49d6-b6ca-4d02dd2db562 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.492614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271134d-2881-49d6-b6ca-4d02dd2db562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.493103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271134d-2881-49d6-b6ca-4d02dd2db562 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.498450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271134d-2881-49d6-b6ca-4d02dd2db562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.501763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271134d-2881-49d6-b6ca-4d02dd2db562 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.568509 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 972/1966] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:41 2022] POST /identity/v3/projects => generated 387 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:41.572600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b29bc-e642-4b5f-9b32-40bae0cfc3bb None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.588994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b29bc-e642-4b5f-9b32-40bae0cfc3bb None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.688831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e762ba8-f1db-4422-8848-b14d383786be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.689594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e762ba8-f1db-4422-8848-b14d383786be admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.689927 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e762ba8-f1db-4422-8848-b14d383786be admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.690547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e762ba8-f1db-4422-8848-b14d383786be admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.691254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e762ba8-f1db-4422-8848-b14d383786be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.695143 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e762ba8-f1db-4422-8848-b14d383786be admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.741776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 995/1967] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:41 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:41.744997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271134d-2881-49d6-b6ca-4d02dd2db562 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.758599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271134d-2881-49d6-b6ca-4d02dd2db562 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.766108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ec7ee-7532-4697-9c52-4c745f1733f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.766790 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ec7ee-7532-4697-9c52-4c745f1733f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.767069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ec7ee-7532-4697-9c52-4c745f1733f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.768175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ec7ee-7532-4697-9c52-4c745f1733f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.769228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ec7ee-7532-4697-9c52-4c745f1733f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.780584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ec7ee-7532-4697-9c52-4c745f1733f5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.799540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 973/1968] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:41.803942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e762ba8-f1db-4422-8848-b14d383786be admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.819470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e762ba8-f1db-4422-8848-b14d383786be admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.828634 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5a123-9371-4e79-b4bc-d0507faab721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.831475 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5a123-9371-4e79-b4bc-d0507faab721 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.831768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5a123-9371-4e79-b4bc-d0507faab721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.832052 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5a123-9371-4e79-b4bc-d0507faab721 admin admin] PATH_INFO: `/v3/projects/933a933873d546b68d260c53a8ee0a96/users/f3b38bf9d7344d33b028a9edbed04c54/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.867498 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5a123-9371-4e79-b4bc-d0507faab721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=933a933873d546b68d260c53a8ee0a96, user_id=f3b38bf9d7344d33b028a9edbed04c54, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.879274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5a123-9371-4e79-b4bc-d0507faab721 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.900916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 996/1969] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:41 2022] PUT /identity/v3/projects/933a933873d546b68d260c53a8ee0a96/users/f3b38bf9d7344d33b028a9edbed04c54/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:41.904673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ec7ee-7532-4697-9c52-4c745f1733f5 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.917018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ec7ee-7532-4697-9c52-4c745f1733f5 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.988049 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3044f-4483-4e77-8d6d-fd1bf3909fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.988744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3044f-4483-4e77-8d6d-fd1bf3909fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.989027 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3044f-4483-4e77-8d6d-fd1bf3909fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.989303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3044f-4483-4e77-8d6d-fd1bf3909fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.990077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3044f-4483-4e77-8d6d-fd1bf3909fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.991774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3044f-4483-4e77-8d6d-fd1bf3909fa0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:42.004603 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 974/1970] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:42.008558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5a123-9371-4e79-b4bc-d0507faab721 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:42.020012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5a123-9371-4e79-b4bc-d0507faab721 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:42.024892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d69eb-9e92-45e8-ad09-f5df258150fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:42.026411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d69eb-9e92-45e8-ad09-f5df258150fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:42.026637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d69eb-9e92-45e8-ad09-f5df258150fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:42.026717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d69eb-9e92-45e8-ad09-f5df258150fb admin admin] PATH_INFO: `/v3/projects/933a933873d546b68d260c53a8ee0a96/users/f3b38bf9d7344d33b028a9edbed04c54/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:42.029444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d69eb-9e92-45e8-ad09-f5df258150fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=933a933873d546b68d260c53a8ee0a96, user_id=f3b38bf9d7344d33b028a9edbed04c54, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:42.032613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d69eb-9e92-45e8-ad09-f5df258150fb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:42.045339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 997/1971] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:42 2022] PUT /identity/v3/projects/933a933873d546b68d260c53a8ee0a96/users/f3b38bf9d7344d33b028a9edbed04c54/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:42.057585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08109bfe-2206-4159-a97e-c0a9e2ea45a5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:42.057918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08109bfe-2206-4159-a97e-c0a9e2ea45a5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:42.058194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08109bfe-2206-4159-a97e-c0a9e2ea45a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:42.063945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:42.092083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-08109bfe-2206-4159-a97e-c0a9e2ea45a5 None None] MFA Rules not processed for user `f3b38bf9d7344d33b028a9edbed04c54`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:42.093636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-08109bfe-2206-4159-a97e-c0a9e2ea45a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:42.170574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 975/1972] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:42.184227 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d69eb-9e92-45e8-ad09-f5df258150fb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:42.190663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e14d69eb-9e92-45e8-ad09-f5df258150fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:42.259139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d69eb-9e92-45e8-ad09-f5df258150fb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:42.263073 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94167ab-d7a4-46fb-bb43-8c293811993b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:42.263663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94167ab-d7a4-46fb-bb43-8c293811993b service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:42.263947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94167ab-d7a4-46fb-bb43-8c293811993b service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:42.264221 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94167ab-d7a4-46fb-bb43-8c293811993b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:42.279060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94167ab-d7a4-46fb-bb43-8c293811993b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:42.280785 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94167ab-d7a4-46fb-bb43-8c293811993b service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:42.305931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 998/1973] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:18:43.335743 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08109bfe-2206-4159-a97e-c0a9e2ea45a5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:43.431447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08109bfe-2206-4159-a97e-c0a9e2ea45a5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:43.436097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:43.436983 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:43.437245 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:43.437501 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] PATH_INFO: `/v3/users/5309a00c37d94746b24f35882f53ebdf` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:43.439195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5309a00c37d94746b24f35882f53ebdf)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:43.442428 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:43.512233 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5309a00c37d94746b24f35882f53ebdf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:43.523292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5309a00c37d94746b24f35882f53ebdf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:43.549446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5309a00c37d94746b24f35882f53ebdf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:43.549446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5309a00c37d94746b24f35882f53ebdf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:43.550672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5309a00c37d94746b24f35882f53ebdf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:43.557812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 976/1974] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:43 2022] DELETE /identity/v3/users/5309a00c37d94746b24f35882f53ebdf => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:43.562588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23da81-97b8-44cf-bd49-712ca40b20c9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:43.563086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23da81-97b8-44cf-bd49-712ca40b20c9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:43.563187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23da81-97b8-44cf-bd49-712ca40b20c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:43.582126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ad23da81-97b8-44cf-bd49-712ca40b20c9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:43.583324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ad23da81-97b8-44cf-bd49-712ca40b20c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:43.640448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:43.656090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 999/1975] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:43.667189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:43.668624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:43.679933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:43.736056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93da8b-c83f-4e7f-8732-811713d2cceb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:43.740618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8c172-2095-4dba-923d-d05d5a909611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:43.741551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8c172-2095-4dba-923d-d05d5a909611 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:43.744144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8c172-2095-4dba-923d-d05d5a909611 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:43.744434 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8c172-2095-4dba-923d-d05d5a909611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:43.757305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8c172-2095-4dba-923d-d05d5a909611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:43.758956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8c172-2095-4dba-923d-d05d5a909611 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:43.770329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 977/1976] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:18:44.058226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23da81-97b8-44cf-bd49-712ca40b20c9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:44.059683 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ad23da81-97b8-44cf-bd49-712ca40b20c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:44.075258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23da81-97b8-44cf-bd49-712ca40b20c9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:44.085009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:44.085903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:44.086219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:44.086522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] PATH_INFO: `/v3/projects/f735a218aee64ec1a8bd43a69b566335` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:44.087630 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f735a218aee64ec1a8bd43a69b566335)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:44.089656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:44.140405 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f735a218aee64ec1a8bd43a69b566335', 'request_id': 'req-afbb510d-aea0-4bfe-a0c0-aade33db5734'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.146766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f735a218aee64ec1a8bd43a69b566335', 'request_id': 'req-afbb510d-aea0-4bfe-a0c0-aade33db5734'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.152437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f735a218aee64ec1a8bd43a69b566335', 'request_id': 'req-afbb510d-aea0-4bfe-a0c0-aade33db5734'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.164722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] The token cache is being invalidate because project f735a218aee64ec1a8bd43a69b566335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:44.165007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.165628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.167195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1000/1977] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:44 2022] DELETE /identity/v3/projects/f735a218aee64ec1a8bd43a69b566335 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:44.171559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8c172-2095-4dba-923d-d05d5a909611 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:44.181810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8c172-2095-4dba-923d-d05d5a909611 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:44.241294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:44.242225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:44.242489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:44.242755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] PATH_INFO: `/v3/users/f3b38bf9d7344d33b028a9edbed04c54` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:44.243956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b38bf9d7344d33b028a9edbed04c54)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:44.245906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:44.302131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b38bf9d7344d33b028a9edbed04c54'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.310614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b38bf9d7344d33b028a9edbed04c54'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.334218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b38bf9d7344d33b028a9edbed04c54'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.334975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b38bf9d7344d33b028a9edbed04c54'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.342127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b38bf9d7344d33b028a9edbed04c54'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.348902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 978/1978] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:44 2022] DELETE /identity/v3/users/f3b38bf9d7344d33b028a9edbed04c54 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:44.442171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:44.443716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:44.461879 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb510d-aea0-4bfe-a0c0-aade33db5734 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:44.536426 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b74683-8c88-412b-a747-cbe592784a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:44.537316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b74683-8c88-412b-a747-cbe592784a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:44.538084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b74683-8c88-412b-a747-cbe592784a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:44.538376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b74683-8c88-412b-a747-cbe592784a80 admin admin] PATH_INFO: `/v3/users/b3885acd272a499ca8d54acc0548e8a7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:44.539592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b74683-8c88-412b-a747-cbe592784a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3885acd272a499ca8d54acc0548e8a7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:44.541555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b74683-8c88-412b-a747-cbe592784a80 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:44.649822 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-33b74683-8c88-412b-a747-cbe592784a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3885acd272a499ca8d54acc0548e8a7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.662983 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:44.665116 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:44.668195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-33b74683-8c88-412b-a747-cbe592784a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3885acd272a499ca8d54acc0548e8a7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.668795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-33b74683-8c88-412b-a747-cbe592784a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3885acd272a499ca8d54acc0548e8a7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.669567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-33b74683-8c88-412b-a747-cbe592784a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3885acd272a499ca8d54acc0548e8a7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.680150 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66832b1e-2a47-4c23-9b63-21d856892843 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:44.696749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64419adb-91e1-41ce-a496-e7f2195ca2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:44.698252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64419adb-91e1-41ce-a496-e7f2195ca2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:44.698527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64419adb-91e1-41ce-a496-e7f2195ca2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:44.698810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64419adb-91e1-41ce-a496-e7f2195ca2b6 admin admin] PATH_INFO: `/v3/projects/933a933873d546b68d260c53a8ee0a96` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:44.699874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-33b74683-8c88-412b-a747-cbe592784a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3885acd272a499ca8d54acc0548e8a7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.700436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64419adb-91e1-41ce-a496-e7f2195ca2b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=933a933873d546b68d260c53a8ee0a96)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:44.703862 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64419adb-91e1-41ce-a496-e7f2195ca2b6 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:44.707712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:44.719294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1001/1979] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:44 2022] DELETE /identity/v3/users/b3885acd272a499ca8d54acc0548e8a7 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:44.725136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb917-2d5e-442f-a42b-21f188c42643 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:44.725428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb917-2d5e-442f-a42b-21f188c42643 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:44.725710 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb917-2d5e-442f-a42b-21f188c42643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:44.737022 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:44.747128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5aacb917-2d5e-442f-a42b-21f188c42643 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:44.748477 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5aacb917-2d5e-442f-a42b-21f188c42643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:44.771078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-64419adb-91e1-41ce-a496-e7f2195ca2b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '933a933873d546b68d260c53a8ee0a96', 'request_id': 'req-64419adb-91e1-41ce-a496-e7f2195ca2b6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.781661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-64419adb-91e1-41ce-a496-e7f2195ca2b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '933a933873d546b68d260c53a8ee0a96', 'request_id': 'req-64419adb-91e1-41ce-a496-e7f2195ca2b6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.789351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-64419adb-91e1-41ce-a496-e7f2195ca2b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '933a933873d546b68d260c53a8ee0a96', 'request_id': 'req-64419adb-91e1-41ce-a496-e7f2195ca2b6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.801481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-64419adb-91e1-41ce-a496-e7f2195ca2b6 admin admin] The token cache is being invalidate because project 933a933873d546b68d260c53a8ee0a96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:44.801809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-64419adb-91e1-41ce-a496-e7f2195ca2b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.802470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-64419adb-91e1-41ce-a496-e7f2195ca2b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.804321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 979/1980] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:44 2022] DELETE /identity/v3/projects/933a933873d546b68d260c53a8ee0a96 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:44.809309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafc5a7-9753-4840-89ee-fd0b6152e6de None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:44.809534 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafc5a7-9753-4840-89ee-fd0b6152e6de None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:44.809846 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafc5a7-9753-4840-89ee-fd0b6152e6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:44.822490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1002/1981] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:44.832358 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-edafc5a7-9753-4840-89ee-fd0b6152e6de None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:44.833572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-edafc5a7-9753-4840-89ee-fd0b6152e6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:44.834802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb917-2d5e-442f-a42b-21f188c42643 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:44.836395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5aacb917-2d5e-442f-a42b-21f188c42643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:44.850212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 980/1982] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:44.851925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e32c60-7833-4c07-be55-45ebd654446c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:44.852210 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e32c60-7833-4c07-be55-45ebd654446c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:44.852482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e32c60-7833-4c07-be55-45ebd654446c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:44.876088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-75e32c60-7833-4c07-be55-45ebd654446c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:44.878147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-75e32c60-7833-4c07-be55-45ebd654446c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:44.888780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 981/1983] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:44.894035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e32c60-7833-4c07-be55-45ebd654446c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:44.905273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e32c60-7833-4c07-be55-45ebd654446c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:44.911438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1d39a-c752-4927-b290-24b418f9c85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:44.912100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1d39a-c752-4927-b290-24b418f9c85a admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:44.912428 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1d39a-c752-4927-b290-24b418f9c85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:44.912699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1d39a-c752-4927-b290-24b418f9c85a admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:44.913413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1d39a-c752-4927-b290-24b418f9c85a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:44.913879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1d39a-c752-4927-b290-24b418f9c85a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:44.915465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1d39a-c752-4927-b290-24b418f9c85a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:44.923891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb917-2d5e-442f-a42b-21f188c42643 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:44.929294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c71163-5910-4806-aa38-41a6b735aaf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:44.930119 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c71163-5910-4806-aa38-41a6b735aaf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:44.930463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c71163-5910-4806-aa38-41a6b735aaf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:44.930822 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c71163-5910-4806-aa38-41a6b735aaf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:44.936900 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 982/1984] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:18:44.941142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c71163-5910-4806-aa38-41a6b735aaf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:44.941294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1d39a-c752-4927-b290-24b418f9c85a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:44.942722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c71163-5910-4806-aa38-41a6b735aaf5 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:44.951727 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1d39a-c752-4927-b290-24b418f9c85a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:44.955561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1003/1985] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:44.958347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0226b-205c-4a76-a24f-10d169b19834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:44.959090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0226b-205c-4a76-a24f-10d169b19834 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:44.959432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0226b-205c-4a76-a24f-10d169b19834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:44.959768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0226b-205c-4a76-a24f-10d169b19834 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:44.960519 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0226b-205c-4a76-a24f-10d169b19834 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:44.962518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0226b-205c-4a76-a24f-10d169b19834 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.002713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 983/1986] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:44 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:45.007124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c71163-5910-4806-aa38-41a6b735aaf5 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.019859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c71163-5910-4806-aa38-41a6b735aaf5 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.096536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb42c9d-533b-4768-accb-764cf3e34aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.097213 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb42c9d-533b-4768-accb-764cf3e34aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.097498 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb42c9d-533b-4768-accb-764cf3e34aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.097800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb42c9d-533b-4768-accb-764cf3e34aa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.098491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb42c9d-533b-4768-accb-764cf3e34aa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.100491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb42c9d-533b-4768-accb-764cf3e34aa4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.127677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1004/1987] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:45 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:45.131802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0226b-205c-4a76-a24f-10d169b19834 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.142138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0226b-205c-4a76-a24f-10d169b19834 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.147194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba96b8-8e6e-47fb-9e3c-223ab140b5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.147858 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba96b8-8e6e-47fb-9e3c-223ab140b5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.148135 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba96b8-8e6e-47fb-9e3c-223ab140b5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.148407 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba96b8-8e6e-47fb-9e3c-223ab140b5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.149150 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba96b8-8e6e-47fb-9e3c-223ab140b5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.150773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba96b8-8e6e-47fb-9e3c-223ab140b5b1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.162393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 984/1988] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:45.166697 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb42c9d-533b-4768-accb-764cf3e34aa4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.178686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb42c9d-533b-4768-accb-764cf3e34aa4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.186511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255784b-a186-455c-80c8-196252a3adc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.187206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255784b-a186-455c-80c8-196252a3adc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.187491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255784b-a186-455c-80c8-196252a3adc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.187769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255784b-a186-455c-80c8-196252a3adc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.190425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255784b-a186-455c-80c8-196252a3adc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.192850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255784b-a186-455c-80c8-196252a3adc7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.208216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1005/1989] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:45.212889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba96b8-8e6e-47fb-9e3c-223ab140b5b1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.225240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba96b8-8e6e-47fb-9e3c-223ab140b5b1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.231909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638055bb-85de-44b2-80e2-f721db61108a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.233964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638055bb-85de-44b2-80e2-f721db61108a admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.234838 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638055bb-85de-44b2-80e2-f721db61108a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.234838 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638055bb-85de-44b2-80e2-f721db61108a admin admin] PATH_INFO: `/v3/projects/58022195ca6946a0a578d182a32e002a/users/d6b91be999a5426f9afb757ff6cc254c/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.252937 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638055bb-85de-44b2-80e2-f721db61108a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58022195ca6946a0a578d182a32e002a, user_id=d6b91be999a5426f9afb757ff6cc254c, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.260588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638055bb-85de-44b2-80e2-f721db61108a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.262080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255784b-a186-455c-80c8-196252a3adc7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.263948 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f255784b-a186-455c-80c8-196252a3adc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:45.279690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 985/1990] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:45 2022] PUT /identity/v3/projects/58022195ca6946a0a578d182a32e002a/users/d6b91be999a5426f9afb757ff6cc254c/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:45.289666 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255784b-a186-455c-80c8-196252a3adc7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.294674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f515-9a31-4c1e-aa02-8d35b355ab15 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.294976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f515-9a31-4c1e-aa02-8d35b355ab15 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.295255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f515-9a31-4c1e-aa02-8d35b355ab15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.309955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:45.334977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-77c5f515-9a31-4c1e-aa02-8d35b355ab15 None None] MFA Rules not processed for user `d6b91be999a5426f9afb757ff6cc254c`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:45.336334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-77c5f515-9a31-4c1e-aa02-8d35b355ab15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:45.379932 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.380810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.381092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.381373 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] PATH_INFO: `/v3/projects/0f4b3a0033364551bed840f854ce09db` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.382622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f4b3a0033364551bed840f854ce09db)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.384571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.422967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 986/1991] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:45.428177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5f515-9a31-4c1e-aa02-8d35b355ab15 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.437790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f4b3a0033364551bed840f854ce09db', 'request_id': 'req-9540df18-4a09-401b-bedd-c827d2864a9f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.439762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5f515-9a31-4c1e-aa02-8d35b355ab15 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.446709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f4b3a0033364551bed840f854ce09db', 'request_id': 'req-9540df18-4a09-401b-bedd-c827d2864a9f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.453576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f4b3a0033364551bed840f854ce09db', 'request_id': 'req-9540df18-4a09-401b-bedd-c827d2864a9f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.456524 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:45.470852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] The token cache is being invalidate because project 0f4b3a0033364551bed840f854ce09db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:45.471310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.472122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.474133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1006/1992] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:45 2022] DELETE /identity/v3/projects/0f4b3a0033364551bed840f854ce09db => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:45.478156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.479498 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:45.522582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7264e-a644-4dd7-901e-a25d8dadae90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.523260 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7264e-a644-4dd7-901e-a25d8dadae90 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.523556 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7264e-a644-4dd7-901e-a25d8dadae90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.523809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7264e-a644-4dd7-901e-a25d8dadae90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.524593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7264e-a644-4dd7-901e-a25d8dadae90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.526968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7264e-a644-4dd7-901e-a25d8dadae90 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.558506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 987/1993] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:45 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:45.562462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7264e-a644-4dd7-901e-a25d8dadae90 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.564688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-20a7264e-a644-4dd7-901e-a25d8dadae90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:45.574456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540df18-4a09-401b-bedd-c827d2864a9f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.624991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:45.629719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7264e-a644-4dd7-901e-a25d8dadae90 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.632916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.634069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.634640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.634880 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5b07e-3905-49ea-9bc9-b4ad688e4709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.635359 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] PATH_INFO: `/v3/users/23ea695e276e44cfb1d483c21aefb71c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.635718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5b07e-3905-49ea-9bc9-b4ad688e4709 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.636002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5b07e-3905-49ea-9bc9-b4ad688e4709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.636278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5b07e-3905-49ea-9bc9-b4ad688e4709 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.636959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5b07e-3905-49ea-9bc9-b4ad688e4709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.637103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=23ea695e276e44cfb1d483c21aefb71c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.638928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5b07e-3905-49ea-9bc9-b4ad688e4709 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.639799 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.659739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 988/1994] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:45 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:45.663827 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5b07e-3905-49ea-9bc9-b4ad688e4709 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.676957 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5b07e-3905-49ea-9bc9-b4ad688e4709 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.683640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fe4aa-20e8-4cff-9e09-7804c75dc529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.684592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fe4aa-20e8-4cff-9e09-7804c75dc529 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.685012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fe4aa-20e8-4cff-9e09-7804c75dc529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.685441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fe4aa-20e8-4cff-9e09-7804c75dc529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.686801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fe4aa-20e8-4cff-9e09-7804c75dc529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.690393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fe4aa-20e8-4cff-9e09-7804c75dc529 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.691678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23ea695e276e44cfb1d483c21aefb71c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.700583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23ea695e276e44cfb1d483c21aefb71c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.701423 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23ea695e276e44cfb1d483c21aefb71c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.702436 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23ea695e276e44cfb1d483c21aefb71c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.704219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 989/1995] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:45.709121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fe4aa-20e8-4cff-9e09-7804c75dc529 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.711206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-858fe4aa-20e8-4cff-9e09-7804c75dc529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:45.718637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23ea695e276e44cfb1d483c21aefb71c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.724594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fe4aa-20e8-4cff-9e09-7804c75dc529 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.729457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9410820-991b-41a7-a94e-7a4933cd5da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.730458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9410820-991b-41a7-a94e-7a4933cd5da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.730743 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9410820-991b-41a7-a94e-7a4933cd5da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.731020 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9410820-991b-41a7-a94e-7a4933cd5da1 admin admin] PATH_INFO: `/v3/projects/56ecf1e505d042feb328aaa9b7fa7b0d/users/394f739f00a446af8ddbd2f4ccef2ea7/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.735060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1007/1996] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:45 2022] DELETE /identity/v3/users/23ea695e276e44cfb1d483c21aefb71c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:45.756700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9410820-991b-41a7-a94e-7a4933cd5da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56ecf1e505d042feb328aaa9b7fa7b0d, user_id=394f739f00a446af8ddbd2f4ccef2ea7, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.760216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9410820-991b-41a7-a94e-7a4933cd5da1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.781350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 990/1997] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:45 2022] PUT /identity/v3/projects/56ecf1e505d042feb328aaa9b7fa7b0d/users/394f739f00a446af8ddbd2f4ccef2ea7/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:45.782466 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.858729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda37b5b-2c0a-4578-a341-e9f880e682ea admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.932913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad2ed0-da86-4bc6-86bc-d72e0078b002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.933589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad2ed0-da86-4bc6-86bc-d72e0078b002 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.933910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad2ed0-da86-4bc6-86bc-d72e0078b002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.934196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad2ed0-da86-4bc6-86bc-d72e0078b002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.935213 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad2ed0-da86-4bc6-86bc-d72e0078b002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.936896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad2ed0-da86-4bc6-86bc-d72e0078b002 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.948329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1008/1998] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:45.952327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9410820-991b-41a7-a94e-7a4933cd5da1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.964955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9410820-991b-41a7-a94e-7a4933cd5da1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.970974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26bb8c-6fe2-4268-a4e4-0424b1a02f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.972320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26bb8c-6fe2-4268-a4e4-0424b1a02f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.972812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26bb8c-6fe2-4268-a4e4-0424b1a02f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.973152 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26bb8c-6fe2-4268-a4e4-0424b1a02f7e admin admin] PATH_INFO: `/v3/projects/56ecf1e505d042feb328aaa9b7fa7b0d/users/394f739f00a446af8ddbd2f4ccef2ea7/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.975513 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26bb8c-6fe2-4268-a4e4-0424b1a02f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=56ecf1e505d042feb328aaa9b7fa7b0d, user_id=394f739f00a446af8ddbd2f4ccef2ea7, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.978469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26bb8c-6fe2-4268-a4e4-0424b1a02f7e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.990473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 991/1999] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:45 2022] PUT /identity/v3/projects/56ecf1e505d042feb328aaa9b7fa7b0d/users/394f739f00a446af8ddbd2f4ccef2ea7/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:46.002191 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100630c-cde4-4bb7-977d-db5ed6671635 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:46.002446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100630c-cde4-4bb7-977d-db5ed6671635 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:46.002706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100630c-cde4-4bb7-977d-db5ed6671635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:46.031485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d100630c-cde4-4bb7-977d-db5ed6671635 None None] MFA Rules not processed for user `394f739f00a446af8ddbd2f4ccef2ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:46.032882 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d100630c-cde4-4bb7-977d-db5ed6671635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:46.037473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26bb8c-6fe2-4268-a4e4-0424b1a02f7e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:46.038437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:46.040223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ac26bb8c-6fe2-4268-a4e4-0424b1a02f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:46.099910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:46.123972 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26bb8c-6fe2-4268-a4e4-0424b1a02f7e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:46.129676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1009/2000] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:46.130129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:46.131060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:46.131412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:46.131755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] PATH_INFO: `/v3/projects/5b187b7c80db4ad5a9b17044aaf381ba` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:46.133014 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b187b7c80db4ad5a9b17044aaf381ba)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:46.135132 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:46.140576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100630c-cde4-4bb7-977d-db5ed6671635 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:46.141977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d100630c-cde4-4bb7-977d-db5ed6671635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:46.183867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b187b7c80db4ad5a9b17044aaf381ba', 'request_id': 'req-7b82e4ca-3569-4696-8492-7513fc23e0a9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.191570 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b187b7c80db4ad5a9b17044aaf381ba', 'request_id': 'req-7b82e4ca-3569-4696-8492-7513fc23e0a9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.197861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b187b7c80db4ad5a9b17044aaf381ba', 'request_id': 'req-7b82e4ca-3569-4696-8492-7513fc23e0a9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.202546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100630c-cde4-4bb7-977d-db5ed6671635 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:46.209113 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] The token cache is being invalidate because project 5b187b7c80db4ad5a9b17044aaf381ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:46.209399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.210180 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.211756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 992/2001] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:46 2022] DELETE /identity/v3/projects/5b187b7c80db4ad5a9b17044aaf381ba => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:46.215838 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:46.217453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:46.252989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fcad3-0c63-42a8-b6d0-92fc6ea6c223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:46.253648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fcad3-0c63-42a8-b6d0-92fc6ea6c223 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:46.253955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fcad3-0c63-42a8-b6d0-92fc6ea6c223 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:46.254227 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fcad3-0c63-42a8-b6d0-92fc6ea6c223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:46.256011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6f9fcad3-0c63-42a8-b6d0-92fc6ea6c223 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:46.284283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82e4ca-3569-4696-8492-7513fc23e0a9 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:46.289012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:46.289912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:46.290204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:46.290481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] PATH_INFO: `/v3/users/112d4a2102ca437587b753965590420d` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:46.291718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=112d4a2102ca437587b753965590420d)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:46.293647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:46.330630 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fcad3-0c63-42a8-b6d0-92fc6ea6c223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:46.332147 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fcad3-0c63-42a8-b6d0-92fc6ea6c223 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:46.342115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1010/2002] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:18:46.342348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '112d4a2102ca437587b753965590420d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.351493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '112d4a2102ca437587b753965590420d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.365042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '112d4a2102ca437587b753965590420d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.365714 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '112d4a2102ca437587b753965590420d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.370155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:46.374308 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '112d4a2102ca437587b753965590420d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.384510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 993/2003] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:46 2022] DELETE /identity/v3/users/112d4a2102ca437587b753965590420d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:46.670207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fcad3-0c63-42a8-b6d0-92fc6ea6c223 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:46.681502 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fcad3-0c63-42a8-b6d0-92fc6ea6c223 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:46.686353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:46.743959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:46.746279 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:46.754962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4d565-f97b-4645-86f7-e77cf0faec3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:46.756364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4d565-f97b-4645-86f7-e77cf0faec3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:46.756650 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4d565-f97b-4645-86f7-e77cf0faec3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:46.756944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4d565-f97b-4645-86f7-e77cf0faec3c admin admin] PATH_INFO: `/v3/projects/c59d02acefb44b9b8651069e9121e02e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:46.758335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4d565-f97b-4645-86f7-e77cf0faec3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c59d02acefb44b9b8651069e9121e02e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:46.760276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4d565-f97b-4645-86f7-e77cf0faec3c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:46.809633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cfc4d565-f97b-4645-86f7-e77cf0faec3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c59d02acefb44b9b8651069e9121e02e', 'request_id': 'req-cfc4d565-f97b-4645-86f7-e77cf0faec3c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.816149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cfc4d565-f97b-4645-86f7-e77cf0faec3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c59d02acefb44b9b8651069e9121e02e', 'request_id': 'req-cfc4d565-f97b-4645-86f7-e77cf0faec3c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.818488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd6bb5-14bb-4aa3-b31f-fda07c787f5c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:46.821993 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cfc4d565-f97b-4645-86f7-e77cf0faec3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c59d02acefb44b9b8651069e9121e02e', 'request_id': 'req-cfc4d565-f97b-4645-86f7-e77cf0faec3c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.835254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cfc4d565-f97b-4645-86f7-e77cf0faec3c admin admin] The token cache is being invalidate because project c59d02acefb44b9b8651069e9121e02e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:46.835544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cfc4d565-f97b-4645-86f7-e77cf0faec3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.836191 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cfc4d565-f97b-4645-86f7-e77cf0faec3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.838177 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1011/2004] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:46 2022] DELETE /identity/v3/projects/c59d02acefb44b9b8651069e9121e02e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:46.843706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed250d-c1ea-496a-b281-a12b6d7beef5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:46.843999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed250d-c1ea-496a-b281-a12b6d7beef5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:46.844282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed250d-c1ea-496a-b281-a12b6d7beef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:46.864060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-57ed250d-c1ea-496a-b281-a12b6d7beef5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:46.865195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-57ed250d-c1ea-496a-b281-a12b6d7beef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:46.878529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2618b-aa31-44eb-a2b2-3251441ffd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:46.879490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2618b-aa31-44eb-a2b2-3251441ffd51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:46.879843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2618b-aa31-44eb-a2b2-3251441ffd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:46.880186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2618b-aa31-44eb-a2b2-3251441ffd51 admin admin] PATH_INFO: `/v3/users/d6b91be999a5426f9afb757ff6cc254c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:46.881456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2618b-aa31-44eb-a2b2-3251441ffd51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b91be999a5426f9afb757ff6cc254c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:46.883506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2618b-aa31-44eb-a2b2-3251441ffd51 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:46.932130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1012/2005] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:46.940320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-edb2618b-aa31-44eb-a2b2-3251441ffd51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b91be999a5426f9afb757ff6cc254c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.941074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddf021-6ac2-4c15-a0c9-a8e258b2aa6e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:46.941369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddf021-6ac2-4c15-a0c9-a8e258b2aa6e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:46.941646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddf021-6ac2-4c15-a0c9-a8e258b2aa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:46.949390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-edb2618b-aa31-44eb-a2b2-3251441ffd51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b91be999a5426f9afb757ff6cc254c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.963323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-0dddf021-6ac2-4c15-a0c9-a8e258b2aa6e None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:46.964376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-edb2618b-aa31-44eb-a2b2-3251441ffd51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b91be999a5426f9afb757ff6cc254c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.964524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0dddf021-6ac2-4c15-a0c9-a8e258b2aa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:46.965074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-edb2618b-aa31-44eb-a2b2-3251441ffd51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b91be999a5426f9afb757ff6cc254c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.973911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-edb2618b-aa31-44eb-a2b2-3251441ffd51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b91be999a5426f9afb757ff6cc254c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:46.978539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1013/2006] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:46.982803 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 994/2007] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:46 2022] DELETE /identity/v3/users/d6b91be999a5426f9afb757ff6cc254c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:46.983095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddf021-6ac2-4c15-a0c9-a8e258b2aa6e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:46.987820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8362f-f72f-433e-8dce-66aaa13452f5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:46.988121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8362f-f72f-433e-8dce-66aaa13452f5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:46.988395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8362f-f72f-433e-8dce-66aaa13452f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:46.994610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddf021-6ac2-4c15-a0c9-a8e258b2aa6e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:47.015446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d0a8362f-f72f-433e-8dce-66aaa13452f5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:47.016668 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d0a8362f-f72f-433e-8dce-66aaa13452f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:47.032145 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:47.043984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:47.096853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38022e93-e980-4ed2-bcbc-34c7a7434f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:47.097899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38022e93-e980-4ed2-bcbc-34c7a7434f9e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:47.098185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38022e93-e980-4ed2-bcbc-34c7a7434f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:47.098468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38022e93-e980-4ed2-bcbc-34c7a7434f9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:47.099262 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38022e93-e980-4ed2-bcbc-34c7a7434f9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:47.099539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38022e93-e980-4ed2-bcbc-34c7a7434f9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:47.101765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38022e93-e980-4ed2-bcbc-34c7a7434f9e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:47.106360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 995/2008] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:47.115346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1014/2009] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:18:47.116978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8362f-f72f-433e-8dce-66aaa13452f5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:47.118393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d0a8362f-f72f-433e-8dce-66aaa13452f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:47.119447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38022e93-e980-4ed2-bcbc-34c7a7434f9e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:47.121085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-38022e93-e980-4ed2-bcbc-34c7a7434f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:47.134024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38022e93-e980-4ed2-bcbc-34c7a7434f9e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:47.138966 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32989771-4a89-4737-b7e2-939a907520ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:47.139628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32989771-4a89-4737-b7e2-939a907520ce admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:47.140043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32989771-4a89-4737-b7e2-939a907520ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:47.140488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32989771-4a89-4737-b7e2-939a907520ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:47.141191 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32989771-4a89-4737-b7e2-939a907520ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:47.144021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32989771-4a89-4737-b7e2-939a907520ce admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:47.178053 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1015/2010] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:47 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:47.179633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32989771-4a89-4737-b7e2-939a907520ce admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:47.184246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8362f-f72f-433e-8dce-66aaa13452f5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:47.232965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53f608-b68a-4e91-93f0-cd95549aacd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:47.233604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53f608-b68a-4e91-93f0-cd95549aacd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:47.233914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53f608-b68a-4e91-93f0-cd95549aacd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:47.234183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53f608-b68a-4e91-93f0-cd95549aacd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:47.243304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53f608-b68a-4e91-93f0-cd95549aacd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:47.243972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32989771-4a89-4737-b7e2-939a907520ce admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:47.244720 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53f608-b68a-4e91-93f0-cd95549aacd0 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:47.248842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f243a26-7d3f-44e6-9a54-5e70b412888d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:47.249507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f243a26-7d3f-44e6-9a54-5e70b412888d admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:47.249834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f243a26-7d3f-44e6-9a54-5e70b412888d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:47.250105 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f243a26-7d3f-44e6-9a54-5e70b412888d admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:47.250851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f243a26-7d3f-44e6-9a54-5e70b412888d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:47.253269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f243a26-7d3f-44e6-9a54-5e70b412888d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:47.254621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 996/2011] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:47.274419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1016/2012] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:47 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:47.278677 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53f608-b68a-4e91-93f0-cd95549aacd0 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:47.288705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53f608-b68a-4e91-93f0-cd95549aacd0 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:47.293337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba3582-af71-4959-a9c5-11bf2afa5ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:47.294025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3582-af71-4959-a9c5-11bf2afa5ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:47.294302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3582-af71-4959-a9c5-11bf2afa5ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:47.294574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3582-af71-4959-a9c5-11bf2afa5ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:47.295521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba3582-af71-4959-a9c5-11bf2afa5ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:47.297401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba3582-af71-4959-a9c5-11bf2afa5ef0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:47.308535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 997/2013] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:47.313532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f243a26-7d3f-44e6-9a54-5e70b412888d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:47.325392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f243a26-7d3f-44e6-9a54-5e70b412888d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:47.330917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39daaf3-f5b4-4ad5-a43b-c3b3c17f788b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:47.331650 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39daaf3-f5b4-4ad5-a43b-c3b3c17f788b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:47.331953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39daaf3-f5b4-4ad5-a43b-c3b3c17f788b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:47.332223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39daaf3-f5b4-4ad5-a43b-c3b3c17f788b admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:47.332941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39daaf3-f5b4-4ad5-a43b-c3b3c17f788b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:47.334636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39daaf3-f5b4-4ad5-a43b-c3b3c17f788b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:47.348797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1017/2014] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:47.350037 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba3582-af71-4959-a9c5-11bf2afa5ef0 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:47.361360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba3582-af71-4959-a9c5-11bf2afa5ef0 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:47.366084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8c1c1-c00d-42e9-b78e-f2a01c290152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:47.366905 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8c1c1-c00d-42e9-b78e-f2a01c290152 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:47.367181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8c1c1-c00d-42e9-b78e-f2a01c290152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:47.367449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8c1c1-c00d-42e9-b78e-f2a01c290152 admin admin] PATH_INFO: `/v3/projects/098290c4499240e5bb671d21c8f42a8a/users/959b31fe78fc4e7583f49714802a927a/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:47.383179 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8c1c1-c00d-42e9-b78e-f2a01c290152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=098290c4499240e5bb671d21c8f42a8a, user_id=959b31fe78fc4e7583f49714802a927a, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:47.385865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8c1c1-c00d-42e9-b78e-f2a01c290152 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:47.400423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 998/2015] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:47 2022] PUT /identity/v3/projects/098290c4499240e5bb671d21c8f42a8a/users/959b31fe78fc4e7583f49714802a927a/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:47.416729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc0e2f-1546-4716-a087-7df0e0197c1d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:47.417027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc0e2f-1546-4716-a087-7df0e0197c1d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:47.417307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc0e2f-1546-4716-a087-7df0e0197c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:47.447431 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b7bc0e2f-1546-4716-a087-7df0e0197c1d None None] MFA Rules not processed for user `959b31fe78fc4e7583f49714802a927a`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:47.448685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b7bc0e2f-1546-4716-a087-7df0e0197c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:47.542125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1018/2016] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:47 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:47.556209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8c1c1-c00d-42e9-b78e-f2a01c290152 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:47.558821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-62a8c1c1-c00d-42e9-b78e-f2a01c290152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:47.619416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:47.619714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc0e2f-1546-4716-a087-7df0e0197c1d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:47.621396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b7bc0e2f-1546-4716-a087-7df0e0197c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:47.633697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8c1c1-c00d-42e9-b78e-f2a01c290152 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:47.637976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:47.638743 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebe390-da0a-438d-87e1-108dc8c2fc58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:47.639457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebe390-da0a-438d-87e1-108dc8c2fc58 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:47.639743 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebe390-da0a-438d-87e1-108dc8c2fc58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:47.640018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebe390-da0a-438d-87e1-108dc8c2fc58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:47.661292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebe390-da0a-438d-87e1-108dc8c2fc58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:47.662804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebe390-da0a-438d-87e1-108dc8c2fc58 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:47.673379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 999/2017] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:47 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:47.715521 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc0e2f-1546-4716-a087-7df0e0197c1d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:47.721806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:47.722787 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:47.723167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:47.723541 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] PATH_INFO: `/v3/projects/58022195ca6946a0a578d182a32e002a` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:47.724844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58022195ca6946a0a578d182a32e002a)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:47.726968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:47.777949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58022195ca6946a0a578d182a32e002a', 'request_id': 'req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:47.785686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58022195ca6946a0a578d182a32e002a', 'request_id': 'req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:47.791825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58022195ca6946a0a578d182a32e002a', 'request_id': 'req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:47.804343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] The token cache is being invalidate because project 58022195ca6946a0a578d182a32e002a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:47.804656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:47.805322 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:47.806968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1019/2018] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:47 2022] DELETE /identity/v3/projects/58022195ca6946a0a578d182a32e002a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:47.811077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebe390-da0a-438d-87e1-108dc8c2fc58 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:47.821910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebe390-da0a-438d-87e1-108dc8c2fc58 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:47.885414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a3766-d8f9-4133-8794-b36eb227b65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:47.886323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a3766-d8f9-4133-8794-b36eb227b65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:47.886599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a3766-d8f9-4133-8794-b36eb227b65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:47.886867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a3766-d8f9-4133-8794-b36eb227b65c admin admin] PATH_INFO: `/v3/users/394f739f00a446af8ddbd2f4ccef2ea7` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:47.888067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a3766-d8f9-4133-8794-b36eb227b65c admin admin] RBAC: Authorizing `identity:delete_user(user_id=394f739f00a446af8ddbd2f4ccef2ea7)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:47.890110 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a3766-d8f9-4133-8794-b36eb227b65c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:47.951451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b38a3766-d8f9-4133-8794-b36eb227b65c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '394f739f00a446af8ddbd2f4ccef2ea7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:47.968530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b38a3766-d8f9-4133-8794-b36eb227b65c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '394f739f00a446af8ddbd2f4ccef2ea7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:47.991711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b38a3766-d8f9-4133-8794-b36eb227b65c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '394f739f00a446af8ddbd2f4ccef2ea7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:47.993491 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b38a3766-d8f9-4133-8794-b36eb227b65c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '394f739f00a446af8ddbd2f4ccef2ea7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:48.003543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b38a3766-d8f9-4133-8794-b36eb227b65c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '394f739f00a446af8ddbd2f4ccef2ea7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:48.010467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1000/2019] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:47 2022] DELETE /identity/v3/users/394f739f00a446af8ddbd2f4ccef2ea7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:48.637105 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:48.639004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:48.678736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3860d-8f53-45c6-9eb9-a456f11f37e1 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:48.777105 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a308d-36c4-46d9-a285-db8169496d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:48.778123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a308d-36c4-46d9-a285-db8169496d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:48.778675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a308d-36c4-46d9-a285-db8169496d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:48.778675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a308d-36c4-46d9-a285-db8169496d6b admin admin] PATH_INFO: `/v3/projects/56ecf1e505d042feb328aaa9b7fa7b0d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:48.779878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a308d-36c4-46d9-a285-db8169496d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=56ecf1e505d042feb328aaa9b7fa7b0d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:48.781844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a308d-36c4-46d9-a285-db8169496d6b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:48.846744 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:48.850215 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e33a308d-36c4-46d9-a285-db8169496d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56ecf1e505d042feb328aaa9b7fa7b0d', 'request_id': 'req-e33a308d-36c4-46d9-a285-db8169496d6b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:48.857849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e33a308d-36c4-46d9-a285-db8169496d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56ecf1e505d042feb328aaa9b7fa7b0d', 'request_id': 'req-e33a308d-36c4-46d9-a285-db8169496d6b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:48.863926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e33a308d-36c4-46d9-a285-db8169496d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56ecf1e505d042feb328aaa9b7fa7b0d', 'request_id': 'req-e33a308d-36c4-46d9-a285-db8169496d6b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:48.878311 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e33a308d-36c4-46d9-a285-db8169496d6b admin admin] The token cache is being invalidate because project 56ecf1e505d042feb328aaa9b7fa7b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:48.878527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e33a308d-36c4-46d9-a285-db8169496d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:48.879196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e33a308d-36c4-46d9-a285-db8169496d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:48.880763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1020/2020] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:48 2022] DELETE /identity/v3/projects/56ecf1e505d042feb328aaa9b7fa7b0d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:48.886012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9502665-6a0a-465a-8fc6-805e1731ebe9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:48.886452 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9502665-6a0a-465a-8fc6-805e1731ebe9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:48.886621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9502665-6a0a-465a-8fc6-805e1731ebe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:48.906042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d9502665-6a0a-465a-8fc6-805e1731ebe9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:48.907242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d9502665-6a0a-465a-8fc6-805e1731ebe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:48.979111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1001/2021] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:48.987365 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e51c8-fc7f-4126-97a2-fa7b8e47e347 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:48.987659 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e51c8-fc7f-4126-97a2-fa7b8e47e347 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:48.987972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e51c8-fc7f-4126-97a2-fa7b8e47e347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:49.008732 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-584e51c8-fc7f-4126-97a2-fa7b8e47e347 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:49.009981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-584e51c8-fc7f-4126-97a2-fa7b8e47e347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:49.020036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1021/2022] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:49.024511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9502665-6a0a-465a-8fc6-805e1731ebe9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:49.036060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9502665-6a0a-465a-8fc6-805e1731ebe9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:49.041690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5592703-cb0b-4399-8ab1-5842a7f7d180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:49.042540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5592703-cb0b-4399-8ab1-5842a7f7d180 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:49.042821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5592703-cb0b-4399-8ab1-5842a7f7d180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:49.043094 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5592703-cb0b-4399-8ab1-5842a7f7d180 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:49.043817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5592703-cb0b-4399-8ab1-5842a7f7d180 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:49.044121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5592703-cb0b-4399-8ab1-5842a7f7d180 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:49.045753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5592703-cb0b-4399-8ab1-5842a7f7d180 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:49.047924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:49.070693 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1002/2023] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:49.071880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e51c8-fc7f-4126-97a2-fa7b8e47e347 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:49.083361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e51c8-fc7f-4126-97a2-fa7b8e47e347 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:49.088434 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba34c9-37b2-4875-863b-b37e985b5213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:49.089138 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba34c9-37b2-4875-863b-b37e985b5213 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:49.089419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba34c9-37b2-4875-863b-b37e985b5213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:49.089697 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba34c9-37b2-4875-863b-b37e985b5213 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:49.090428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba34c9-37b2-4875-863b-b37e985b5213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:49.092765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba34c9-37b2-4875-863b-b37e985b5213 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:49.133658 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1022/2024] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:49 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:49.135105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5592703-cb0b-4399-8ab1-5842a7f7d180 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:49.148070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5592703-cb0b-4399-8ab1-5842a7f7d180 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:49.217755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fb645-b00b-4d18-b6c7-3d7f78d11240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:49.217755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fb645-b00b-4d18-b6c7-3d7f78d11240 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:49.217755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fb645-b00b-4d18-b6c7-3d7f78d11240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:49.217755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fb645-b00b-4d18-b6c7-3d7f78d11240 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:49.218264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fb645-b00b-4d18-b6c7-3d7f78d11240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:49.219768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fb645-b00b-4d18-b6c7-3d7f78d11240 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:49.242178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1003/2025] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:49 2022] POST /identity/v3/users => generated 284 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:49.246378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba34c9-37b2-4875-863b-b37e985b5213 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:49.256975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba34c9-37b2-4875-863b-b37e985b5213 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:49.261871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bb373-bebc-402b-8fd1-fef912283805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:49.262607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bb373-bebc-402b-8fd1-fef912283805 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:49.262955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bb373-bebc-402b-8fd1-fef912283805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:49.263312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bb373-bebc-402b-8fd1-fef912283805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:49.264206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bb373-bebc-402b-8fd1-fef912283805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:49.265907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bb373-bebc-402b-8fd1-fef912283805 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:49.278339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1023/2026] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:49.282119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fb645-b00b-4d18-b6c7-3d7f78d11240 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:49.293978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fb645-b00b-4d18-b6c7-3d7f78d11240 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:49.299195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642d3be-43dc-42e5-9a2e-7daa57c77c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:49.299876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642d3be-43dc-42e5-9a2e-7daa57c77c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:49.300159 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642d3be-43dc-42e5-9a2e-7daa57c77c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:49.300436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642d3be-43dc-42e5-9a2e-7daa57c77c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:49.301155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642d3be-43dc-42e5-9a2e-7daa57c77c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:49.302909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642d3be-43dc-42e5-9a2e-7daa57c77c3f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:49.315332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1004/2027] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:49.320185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bb373-bebc-402b-8fd1-fef912283805 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:49.331881 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bb373-bebc-402b-8fd1-fef912283805 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:49.336523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344f55b-ed90-43f1-8cb5-044a4c9b869b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:49.337438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344f55b-ed90-43f1-8cb5-044a4c9b869b admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:49.337719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344f55b-ed90-43f1-8cb5-044a4c9b869b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:49.338023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344f55b-ed90-43f1-8cb5-044a4c9b869b admin admin] PATH_INFO: `/v3/projects/2df16509d14441cabe3fb9c7806a2639/users/3dfa4ee464054e3e87c3f4784b8931a2/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:49.353145 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344f55b-ed90-43f1-8cb5-044a4c9b869b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df16509d14441cabe3fb9c7806a2639, user_id=3dfa4ee464054e3e87c3f4784b8931a2, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:49.355854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344f55b-ed90-43f1-8cb5-044a4c9b869b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:49.369995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1024/2028] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:49 2022] PUT /identity/v3/projects/2df16509d14441cabe3fb9c7806a2639/users/3dfa4ee464054e3e87c3f4784b8931a2/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:49.381453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3a701-65c3-4188-bc45-d03eef28c0de None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:49.381667 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3a701-65c3-4188-bc45-d03eef28c0de None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:49.381976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3a701-65c3-4188-bc45-d03eef28c0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:49.411679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-93e3a701-65c3-4188-bc45-d03eef28c0de None None] MFA Rules not processed for user `3dfa4ee464054e3e87c3f4784b8931a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:49.412917 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-93e3a701-65c3-4188-bc45-d03eef28c0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:49.483197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1005/2029] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:49.503632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344f55b-ed90-43f1-8cb5-044a4c9b869b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:49.505381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8344f55b-ed90-43f1-8cb5-044a4c9b869b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:49.593539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344f55b-ed90-43f1-8cb5-044a4c9b869b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:49.599962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bac97-4435-497a-8e67-2150d4e4f51d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a67120485f74f8bb0ff188cd3abd21c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:49.600623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bac97-4435-497a-8e67-2150d4e4f51d service glance] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:49.600916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bac97-4435-497a-8e67-2150d4e4f51d service glance] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:49.601182 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bac97-4435-497a-8e67-2150d4e4f51d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:49.616401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bac97-4435-497a-8e67-2150d4e4f51d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:49.617904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bac97-4435-497a-8e67-2150d4e4f51d service glance] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:49.628504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1025/2030] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:50.596514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1deed3b-3939-4e60-a72c-870b342c424b None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:50.597196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1deed3b-3939-4e60-a72c-870b342c424b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:50.597196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1deed3b-3939-4e60-a72c-870b342c424b None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:50.598729 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1006/2031] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:18:50.605174 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adb505-a8be-4576-aed2-d1f041004b9e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:50.605876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adb505-a8be-4576-aed2-d1f041004b9e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:50.605876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adb505-a8be-4576-aed2-d1f041004b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:50.625204 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-86adb505-a8be-4576-aed2-d1f041004b9e None None] MFA Rules not processed for user `7ee40bcb7905490c9ad3e39e7a93b122`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:50.627594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-86adb505-a8be-4576-aed2-d1f041004b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:50.761452 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1026/2032] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:18:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:50.774540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1deed3b-3939-4e60-a72c-870b342c424b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:50.776982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e1deed3b-3939-4e60-a72c-870b342c424b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:50.800808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1deed3b-3939-4e60-a72c-870b342c424b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:50.809046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db7982-434b-4c0c-b5e6-50f4d7e4c093 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:50.809879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db7982-434b-4c0c-b5e6-50f4d7e4c093 service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:50.810280 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db7982-434b-4c0c-b5e6-50f4d7e4c093 service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:50.810637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db7982-434b-4c0c-b5e6-50f4d7e4c093 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:50.823587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db7982-434b-4c0c-b5e6-50f4d7e4c093 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:50.826198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db7982-434b-4c0c-b5e6-50f4d7e4c093 service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:50.851295 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1007/2033] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:51.828397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adb505-a8be-4576-aed2-d1f041004b9e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:51.830762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-86adb505-a8be-4576-aed2-d1f041004b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:51.841660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:51.912156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adb505-a8be-4576-aed2-d1f041004b9e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:51.917592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac930e7-3a85-4907-a39d-3cb5151e3c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:51.918900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac930e7-3a85-4907-a39d-3cb5151e3c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:51.919507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac930e7-3a85-4907-a39d-3cb5151e3c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:51.919963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac930e7-3a85-4907-a39d-3cb5151e3c47 admin admin] PATH_INFO: `/v3/users/d7d44386436c4dbfa66aeeb5a9b04a71` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:51.921659 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac930e7-3a85-4907-a39d-3cb5151e3c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d44386436c4dbfa66aeeb5a9b04a71)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:51.924137 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac930e7-3a85-4907-a39d-3cb5151e3c47 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:51.960679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db7982-434b-4c0c-b5e6-50f4d7e4c093 service placement] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:51.962900 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d2db7982-434b-4c0c-b5e6-50f4d7e4c093 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:51.998292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db7982-434b-4c0c-b5e6-50f4d7e4c093 service placement] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.009163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.010070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.010521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.010793 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] PATH_INFO: `/v3/users/7c394836f8214070b0eda102a6cc9cd4` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.012097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c394836f8214070b0eda102a6cc9cd4)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.014221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.022640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-eac930e7-3a85-4907-a39d-3cb5151e3c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d44386436c4dbfa66aeeb5a9b04a71'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.032260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-eac930e7-3a85-4907-a39d-3cb5151e3c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d44386436c4dbfa66aeeb5a9b04a71'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.033327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-eac930e7-3a85-4907-a39d-3cb5151e3c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d44386436c4dbfa66aeeb5a9b04a71'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.034050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-eac930e7-3a85-4907-a39d-3cb5151e3c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d44386436c4dbfa66aeeb5a9b04a71'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.059648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-eac930e7-3a85-4907-a39d-3cb5151e3c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d44386436c4dbfa66aeeb5a9b04a71'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.074125 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:52.076383 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1027/2034] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:51 2022] DELETE /identity/v3/users/d7d44386436c4dbfa66aeeb5a9b04a71 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:52.076971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c394836f8214070b0eda102a6cc9cd4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.082248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618254f-0b03-4d1b-acb5-e6efc96669c9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.082739 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618254f-0b03-4d1b-acb5-e6efc96669c9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.083068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618254f-0b03-4d1b-acb5-e6efc96669c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.091763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c394836f8214070b0eda102a6cc9cd4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.109428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5618254f-0b03-4d1b-acb5-e6efc96669c9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:52.110717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5618254f-0b03-4d1b-acb5-e6efc96669c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:52.111844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c394836f8214070b0eda102a6cc9cd4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.112798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c394836f8214070b0eda102a6cc9cd4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.120694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c394836f8214070b0eda102a6cc9cd4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.127674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1008/2035] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:51 2022] DELETE /identity/v3/users/7c394836f8214070b0eda102a6cc9cd4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:52.178896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1028/2036] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:52.188171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.190986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:52.262086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b324ac-0e23-4875-bd86-03ab83d77f7a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.269842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830125a-6e77-4c44-84eb-db30791aaaff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.269842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830125a-6e77-4c44-84eb-db30791aaaff service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.270104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830125a-6e77-4c44-84eb-db30791aaaff service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.270239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830125a-6e77-4c44-84eb-db30791aaaff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.322851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618254f-0b03-4d1b-acb5-e6efc96669c9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.324344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5618254f-0b03-4d1b-acb5-e6efc96669c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:52.345152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618254f-0b03-4d1b-acb5-e6efc96669c9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.379176 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:52.381958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830125a-6e77-4c44-84eb-db30791aaaff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.384682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830125a-6e77-4c44-84eb-db30791aaaff service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.401984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1009/2037] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:52.431006 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.431882 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.432195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.432472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] PATH_INFO: `/v3/projects/cfb4559124a04485a412722864f5dc5f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.434013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfb4559124a04485a412722864f5dc5f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.437752 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.505407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb4559124a04485a412722864f5dc5f', 'request_id': 'req-0e182f7c-10fb-4b82-9835-2cc418e6803a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.512839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb4559124a04485a412722864f5dc5f', 'request_id': 'req-0e182f7c-10fb-4b82-9835-2cc418e6803a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.523987 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb4559124a04485a412722864f5dc5f', 'request_id': 'req-0e182f7c-10fb-4b82-9835-2cc418e6803a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.540592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] The token cache is being invalidate because project cfb4559124a04485a412722864f5dc5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:52.540914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.541607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.547673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830125a-6e77-4c44-84eb-db30791aaaff service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.557813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1029/2038] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:52 2022] DELETE /identity/v3/projects/cfb4559124a04485a412722864f5dc5f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:52.564955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830125a-6e77-4c44-84eb-db30791aaaff service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.652544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.653454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.653751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.654022 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] PATH_INFO: `/v3/users/d93bf5ee781440c8806d4f77d9d45afc` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.655250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d93bf5ee781440c8806d4f77d9d45afc)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.657153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.671710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:52.738762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd93bf5ee781440c8806d4f77d9d45afc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.751731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd93bf5ee781440c8806d4f77d9d45afc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.765151 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd93bf5ee781440c8806d4f77d9d45afc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.766277 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd93bf5ee781440c8806d4f77d9d45afc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.775023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd93bf5ee781440c8806d4f77d9d45afc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:52.781414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1010/2039] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:52 2022] DELETE /identity/v3/users/d93bf5ee781440c8806d4f77d9d45afc => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:52.932544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.934184 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:52.948429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e182f7c-10fb-4b82-9835-2cc418e6803a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:53.062663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:53.064503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:53.078364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:53.080020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.080311 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.080591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] PATH_INFO: `/v3/projects/8c9683df298148b1a0fa8bb446fb7e45` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.081915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c9683df298148b1a0fa8bb446fb7e45)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:53.083927 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:53.166690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c9683df298148b1a0fa8bb446fb7e45', 'request_id': 'req-d0184cb6-8965-4a4c-9866-a3e66530d579'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.174931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c9683df298148b1a0fa8bb446fb7e45', 'request_id': 'req-d0184cb6-8965-4a4c-9866-a3e66530d579'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.188803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:53.191617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c9683df298148b1a0fa8bb446fb7e45', 'request_id': 'req-d0184cb6-8965-4a4c-9866-a3e66530d579'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.194730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff45c4-4c51-4bd0-a536-daaa898424c4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:53.211687 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] The token cache is being invalidate because project 8c9683df298148b1a0fa8bb446fb7e45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:53.211983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.212717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.217975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1030/2040] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:52 2022] DELETE /identity/v3/projects/8c9683df298148b1a0fa8bb446fb7e45 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:53.219103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:53.221038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:53.280321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:53.281540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.281787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.282045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a admin admin] PATH_INFO: `/v3/projects/db7cf079fb8e4910afa6e4639616bef9` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.284066 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=db7cf079fb8e4910afa6e4639616bef9)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:53.286394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:53.317886 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0184cb6-8965-4a4c-9866-a3e66530d579 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:53.324998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1f82d-e6f9-4ac7-b47c-b4176a2ce429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:53.326015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1f82d-e6f9-4ac7-b47c-b4176a2ce429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.326385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1f82d-e6f9-4ac7-b47c-b4176a2ce429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.326726 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1f82d-e6f9-4ac7-b47c-b4176a2ce429 admin admin] PATH_INFO: `/v3/users/2c8c1c052f3648e0b11b732974b170e3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.328156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1f82d-e6f9-4ac7-b47c-b4176a2ce429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c8c1c052f3648e0b11b732974b170e3)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:53.331848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1f82d-e6f9-4ac7-b47c-b4176a2ce429 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:53.343500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:53.354040 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db7cf079fb8e4910afa6e4639616bef9', 'request_id': 'req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.363717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db7cf079fb8e4910afa6e4639616bef9', 'request_id': 'req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.371190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db7cf079fb8e4910afa6e4639616bef9', 'request_id': 'req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.399595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-13e1f82d-e6f9-4ac7-b47c-b4176a2ce429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c8c1c052f3648e0b11b732974b170e3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.400127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a admin admin] The token cache is being invalidate because project db7cf079fb8e4910afa6e4639616bef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:53.400627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.401954 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3e22e9c3-a55b-4e73-9902-a16a22da4a4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.405044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1011/2041] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:53 2022] DELETE /identity/v3/projects/db7cf079fb8e4910afa6e4639616bef9 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:53.408497 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-13e1f82d-e6f9-4ac7-b47c-b4176a2ce429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c8c1c052f3648e0b11b732974b170e3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.409484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-13e1f82d-e6f9-4ac7-b47c-b4176a2ce429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c8c1c052f3648e0b11b732974b170e3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.410388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-13e1f82d-e6f9-4ac7-b47c-b4176a2ce429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c8c1c052f3648e0b11b732974b170e3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.411256 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d8582-7cb7-4a60-a006-0a09afb8818f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.411555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d8582-7cb7-4a60-a006-0a09afb8818f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.411851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d8582-7cb7-4a60-a006-0a09afb8818f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.427511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-13e1f82d-e6f9-4ac7-b47c-b4176a2ce429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c8c1c052f3648e0b11b732974b170e3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.441835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1031/2042] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:53 2022] DELETE /identity/v3/users/2c8c1c052f3648e0b11b732974b170e3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:53.450750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-338d8582-7cb7-4a60-a006-0a09afb8818f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:53.452364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-338d8582-7cb7-4a60-a006-0a09afb8818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:53.569992 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1012/2043] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:53.573550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5bd12-1df7-4fba-aca1-83e77ba24184 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.573867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5bd12-1df7-4fba-aca1-83e77ba24184 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.574788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5bd12-1df7-4fba-aca1-83e77ba24184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.598330 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a3c5bd12-1df7-4fba-aca1-83e77ba24184 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:53.599545 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a3c5bd12-1df7-4fba-aca1-83e77ba24184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:53.609516 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1032/2044] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:53.616807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d8582-7cb7-4a60-a006-0a09afb8818f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:53.627222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d8582-7cb7-4a60-a006-0a09afb8818f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:53.632318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe8cf7-dc75-4984-b6a2-86bfb486c173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:53.632987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe8cf7-dc75-4984-b6a2-86bfb486c173 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.633414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe8cf7-dc75-4984-b6a2-86bfb486c173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.633696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe8cf7-dc75-4984-b6a2-86bfb486c173 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.634459 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe8cf7-dc75-4984-b6a2-86bfb486c173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:53.634792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe8cf7-dc75-4984-b6a2-86bfb486c173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:53.636640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe8cf7-dc75-4984-b6a2-86bfb486c173 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:53.649007 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1013/2045] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:53.653411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5bd12-1df7-4fba-aca1-83e77ba24184 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:53.663768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5bd12-1df7-4fba-aca1-83e77ba24184 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:53.668933 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78d60a-d61c-40e2-9525-994e2e5c9504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:53.669666 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78d60a-d61c-40e2-9525-994e2e5c9504 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.670117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78d60a-d61c-40e2-9525-994e2e5c9504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.670456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78d60a-d61c-40e2-9525-994e2e5c9504 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.671454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78d60a-d61c-40e2-9525-994e2e5c9504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:53.674123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78d60a-d61c-40e2-9525-994e2e5c9504 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:53.702286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1033/2046] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:53 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:53.706385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe8cf7-dc75-4984-b6a2-86bfb486c173 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:53.716802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe8cf7-dc75-4984-b6a2-86bfb486c173 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:53.779751 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78d60a-d61c-40e2-9525-994e2e5c9504 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:53.786951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3f78d60a-d61c-40e2-9525-994e2e5c9504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:53.810008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee231d15-106f-4339-89e4-23dc97193412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:53.810971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee231d15-106f-4339-89e4-23dc97193412 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.811422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee231d15-106f-4339-89e4-23dc97193412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.811764 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee231d15-106f-4339-89e4-23dc97193412 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.812780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee231d15-106f-4339-89e4-23dc97193412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:53.815524 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee231d15-106f-4339-89e4-23dc97193412 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:53.852470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1014/2047] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:53 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:53.856499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee231d15-106f-4339-89e4-23dc97193412 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:53.867966 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee231d15-106f-4339-89e4-23dc97193412 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:53.876831 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef9a7c-6dc4-42a8-b16b-dc367699594b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:53.878088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef9a7c-6dc4-42a8-b16b-dc367699594b admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.878689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef9a7c-6dc4-42a8-b16b-dc367699594b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.879246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef9a7c-6dc4-42a8-b16b-dc367699594b admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.881256 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef9a7c-6dc4-42a8-b16b-dc367699594b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:53.881566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:53.885182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef9a7c-6dc4-42a8-b16b-dc367699594b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:53.898728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1015/2048] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:53.903149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef9a7c-6dc4-42a8-b16b-dc367699594b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:53.916581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef9a7c-6dc4-42a8-b16b-dc367699594b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:53.924169 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78d60a-d61c-40e2-9525-994e2e5c9504 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:53.932907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:53.935196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.935481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.935765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b admin admin] PATH_INFO: `/v3/projects/457ad2bab3764ef8a2bd92949e393b97` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.937023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=457ad2bab3764ef8a2bd92949e393b97)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:53.939098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:53.945307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1b76d-71d6-446e-b9ed-871eef0b2c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:53.946329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1b76d-71d6-446e-b9ed-871eef0b2c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.946798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1b76d-71d6-446e-b9ed-871eef0b2c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.947288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1b76d-71d6-446e-b9ed-871eef0b2c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.948189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1b76d-71d6-446e-b9ed-871eef0b2c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:53.951571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1b76d-71d6-446e-b9ed-871eef0b2c23 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:53.967786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1016/2049] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:53.973539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1b76d-71d6-446e-b9ed-871eef0b2c23 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:53.994159 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '457ad2bab3764ef8a2bd92949e393b97', 'request_id': 'req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:53.997405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1b76d-71d6-446e-b9ed-871eef0b2c23 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.001193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '457ad2bab3764ef8a2bd92949e393b97', 'request_id': 'req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:54.008012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '457ad2bab3764ef8a2bd92949e393b97', 'request_id': 'req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:54.024046 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b admin admin] The token cache is being invalidate because project 457ad2bab3764ef8a2bd92949e393b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:54.024486 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:54.026279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0f3420fc-09d3-4b61-8dfb-16cbd40e0c0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:54.031645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1034/2050] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:53 2022] DELETE /identity/v3/projects/457ad2bab3764ef8a2bd92949e393b97 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:54.033967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85523620-b567-4e79-8571-d552d56da30f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.034256 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85523620-b567-4e79-8571-d552d56da30f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.034576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85523620-b567-4e79-8571-d552d56da30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.043234 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:54.056417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-85523620-b567-4e79-8571-d552d56da30f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:54.057630 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-85523620-b567-4e79-8571-d552d56da30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:54.090940 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa209d-6f91-4481-bdad-8baf357a3a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.091859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa209d-6f91-4481-bdad-8baf357a3a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.092129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa209d-6f91-4481-bdad-8baf357a3a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.092402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa209d-6f91-4481-bdad-8baf357a3a65 admin admin] PATH_INFO: `/v3/projects/b6429497772e4503836688b7a341bba4/users/f419e66145de47ffad7bc23fd606434e/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.114609 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa209d-6f91-4481-bdad-8baf357a3a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6429497772e4503836688b7a341bba4, user_id=f419e66145de47ffad7bc23fd606434e, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.117489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa209d-6f91-4481-bdad-8baf357a3a65 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.133646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1017/2051] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:53 2022] PUT /identity/v3/projects/b6429497772e4503836688b7a341bba4/users/f419e66145de47ffad7bc23fd606434e/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:54.139447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1035/2052] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:54.144847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713ab19-1202-4bb1-ad60-4024db8f9783 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.145134 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713ab19-1202-4bb1-ad60-4024db8f9783 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.145414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713ab19-1202-4bb1-ad60-4024db8f9783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.148071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3604a-ea37-4352-b67d-b18cf2677253 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.148502 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3604a-ea37-4352-b67d-b18cf2677253 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.148832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3604a-ea37-4352-b67d-b18cf2677253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.172302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-63a3604a-ea37-4352-b67d-b18cf2677253 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:54.174330 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-63a3604a-ea37-4352-b67d-b18cf2677253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:54.179981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0713ab19-1202-4bb1-ad60-4024db8f9783 None None] MFA Rules not processed for user `f419e66145de47ffad7bc23fd606434e`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:54.181215 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0713ab19-1202-4bb1-ad60-4024db8f9783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:54.248885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1036/2053] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:54.253076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3604a-ea37-4352-b67d-b18cf2677253 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.262468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3604a-ea37-4352-b67d-b18cf2677253 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.267364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de5d49-7342-4711-9f82-3097c59060c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.268203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de5d49-7342-4711-9f82-3097c59060c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.268470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de5d49-7342-4711-9f82-3097c59060c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.268765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de5d49-7342-4711-9f82-3097c59060c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.269143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1018/2054] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:54.270556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de5d49-7342-4711-9f82-3097c59060c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:54.270950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de5d49-7342-4711-9f82-3097c59060c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.273070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de5d49-7342-4711-9f82-3097c59060c3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.273467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713ab19-1202-4bb1-ad60-4024db8f9783 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.274808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0713ab19-1202-4bb1-ad60-4024db8f9783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:54.280821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:54.288810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1037/2055] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:18:54.289458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713ab19-1202-4bb1-ad60-4024db8f9783 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.292965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de5d49-7342-4711-9f82-3097c59060c3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.294142 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027acfde-3c07-44f9-9b7f-2d4ba167e07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.294881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027acfde-3c07-44f9-9b7f-2d4ba167e07f admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.295199 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027acfde-3c07-44f9-9b7f-2d4ba167e07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.295475 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027acfde-3c07-44f9-9b7f-2d4ba167e07f admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.296189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027acfde-3c07-44f9-9b7f-2d4ba167e07f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.299019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027acfde-3c07-44f9-9b7f-2d4ba167e07f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.305268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de5d49-7342-4711-9f82-3097c59060c3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.310278 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7a0eb-db3a-42ba-a8f6-4ca1e604bd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.310931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7a0eb-db3a-42ba-a8f6-4ca1e604bd6e admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.311215 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7a0eb-db3a-42ba-a8f6-4ca1e604bd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.311495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7a0eb-db3a-42ba-a8f6-4ca1e604bd6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.312184 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7a0eb-db3a-42ba-a8f6-4ca1e604bd6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.314273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7a0eb-db3a-42ba-a8f6-4ca1e604bd6e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.329154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1019/2056] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:54 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:54.333006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027acfde-3c07-44f9-9b7f-2d4ba167e07f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.351195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1038/2057] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:54 2022] POST /identity/v3/projects => generated 357 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:54.355368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7a0eb-db3a-42ba-a8f6-4ca1e604bd6e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.411386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027acfde-3c07-44f9-9b7f-2d4ba167e07f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.428959 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7a0eb-db3a-42ba-a8f6-4ca1e604bd6e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.433144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:54.435293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c14f04-c75a-48eb-ac26-6e101425a579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.436152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c14f04-c75a-48eb-ac26-6e101425a579 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.436478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c14f04-c75a-48eb-ac26-6e101425a579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.436763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c14f04-c75a-48eb-ac26-6e101425a579 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.437574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c14f04-c75a-48eb-ac26-6e101425a579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.441044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c14f04-c75a-48eb-ac26-6e101425a579 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.465080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1039/2058] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:54 2022] POST /identity/v3/users => generated 279 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:54.469699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c14f04-c75a-48eb-ac26-6e101425a579 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.480990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575a102-a827-4c85-9b87-51f8da842e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.481465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c14f04-c75a-48eb-ac26-6e101425a579 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.481592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575a102-a827-4c85-9b87-51f8da842e27 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.481938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575a102-a827-4c85-9b87-51f8da842e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.482235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575a102-a827-4c85-9b87-51f8da842e27 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.482856 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575a102-a827-4c85-9b87-51f8da842e27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.484759 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575a102-a827-4c85-9b87-51f8da842e27 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.487566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cbcb5-ec63-4247-82b7-764d95a3aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.488236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cbcb5-ec63-4247-82b7-764d95a3aa6d admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.488539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cbcb5-ec63-4247-82b7-764d95a3aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.488790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cbcb5-ec63-4247-82b7-764d95a3aa6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.489510 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6cbcb5-ec63-4247-82b7-764d95a3aa6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.491151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6cbcb5-ec63-4247-82b7-764d95a3aa6d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.503285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1040/2059] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:54.506580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cbcb5-ec63-4247-82b7-764d95a3aa6d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.507535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1020/2060] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:54 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:54.511418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575a102-a827-4c85-9b87-51f8da842e27 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.519307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cbcb5-ec63-4247-82b7-764d95a3aa6d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.523028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575a102-a827-4c85-9b87-51f8da842e27 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.524693 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1d649-f951-499a-a1f5-da512eb06126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.525413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1d649-f951-499a-a1f5-da512eb06126 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.525702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1d649-f951-499a-a1f5-da512eb06126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.526080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1d649-f951-499a-a1f5-da512eb06126 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.526970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1d649-f951-499a-a1f5-da512eb06126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.528437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103683bb-f699-49cb-a7e7-7617764c12c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.528615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1d649-f951-499a-a1f5-da512eb06126 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.529307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103683bb-f699-49cb-a7e7-7617764c12c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.529651 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103683bb-f699-49cb-a7e7-7617764c12c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.529997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103683bb-f699-49cb-a7e7-7617764c12c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.530713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103683bb-f699-49cb-a7e7-7617764c12c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.532322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103683bb-f699-49cb-a7e7-7617764c12c1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.543443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1021/2061] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:54.545348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103683bb-f699-49cb-a7e7-7617764c12c1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.545907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1041/2062] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:54.547940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1d649-f951-499a-a1f5-da512eb06126 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.559810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103683bb-f699-49cb-a7e7-7617764c12c1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.559909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1d649-f951-499a-a1f5-da512eb06126 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.563118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a469679-9456-4187-90a1-127e064bd099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.564480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a469679-9456-4187-90a1-127e064bd099 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.564590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f709d5-77c7-4be5-bdee-473b52309c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.564783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a469679-9456-4187-90a1-127e064bd099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.565114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a469679-9456-4187-90a1-127e064bd099 admin admin] PATH_INFO: `/v3/projects/dc08a57d53904aae84bfd5df87948cee/users/d35af6eee793430d9d3b46599f807c17/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.565854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f709d5-77c7-4be5-bdee-473b52309c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.566302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f709d5-77c7-4be5-bdee-473b52309c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.566468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f709d5-77c7-4be5-bdee-473b52309c3e admin admin] PATH_INFO: `/v3/projects/f31fad7f228d455d92ca3e6ab12458f9/users/47db4ef4868e446890f9e5394ef6fbb1/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.587244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a469679-9456-4187-90a1-127e064bd099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc08a57d53904aae84bfd5df87948cee, user_id=d35af6eee793430d9d3b46599f807c17, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.587576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f709d5-77c7-4be5-bdee-473b52309c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f31fad7f228d455d92ca3e6ab12458f9, user_id=47db4ef4868e446890f9e5394ef6fbb1, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.590058 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a469679-9456-4187-90a1-127e064bd099 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.590366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f709d5-77c7-4be5-bdee-473b52309c3e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.608416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1022/2063] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:54 2022] PUT /identity/v3/projects/dc08a57d53904aae84bfd5df87948cee/users/d35af6eee793430d9d3b46599f807c17/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:54.608689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1042/2064] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:54 2022] PUT /identity/v3/projects/f31fad7f228d455d92ca3e6ab12458f9/users/47db4ef4868e446890f9e5394ef6fbb1/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:54.619917 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683b27a-0816-4319-a4c8-ad43c4f2b3e0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.620076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683b27a-0816-4319-a4c8-ad43c4f2b3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.620441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683b27a-0816-4319-a4c8-ad43c4f2b3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.621815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f709d5-77c7-4be5-bdee-473b52309c3e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.649238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a683b27a-0816-4319-a4c8-ad43c4f2b3e0 None None] MFA Rules not processed for user `d35af6eee793430d9d3b46599f807c17`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:54.650497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a683b27a-0816-4319-a4c8-ad43c4f2b3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:54.692139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f709d5-77c7-4be5-bdee-473b52309c3e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.697255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f498a5-c1cb-4904-bcca-7973bc40107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.697976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f498a5-c1cb-4904-bcca-7973bc40107b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.698258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f498a5-c1cb-4904-bcca-7973bc40107b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.698528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f498a5-c1cb-4904-bcca-7973bc40107b admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.699459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f498a5-c1cb-4904-bcca-7973bc40107b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.701565 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f498a5-c1cb-4904-bcca-7973bc40107b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.712458 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1043/2065] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:54.716940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f498a5-c1cb-4904-bcca-7973bc40107b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.728441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f498a5-c1cb-4904-bcca-7973bc40107b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.733481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a09ad7-36d8-4a54-bbad-d587afa86449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.734384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a09ad7-36d8-4a54-bbad-d587afa86449 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.734665 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a09ad7-36d8-4a54-bbad-d587afa86449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.734943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a09ad7-36d8-4a54-bbad-d587afa86449 admin admin] PATH_INFO: `/v3/projects/f31fad7f228d455d92ca3e6ab12458f9/users/47db4ef4868e446890f9e5394ef6fbb1/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.737041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a09ad7-36d8-4a54-bbad-d587afa86449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f31fad7f228d455d92ca3e6ab12458f9, user_id=47db4ef4868e446890f9e5394ef6fbb1, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.739832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a09ad7-36d8-4a54-bbad-d587afa86449 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.745305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1023/2066] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:54 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:54.749858 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683b27a-0816-4319-a4c8-ad43c4f2b3e0 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.753850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1044/2067] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:54 2022] PUT /identity/v3/projects/f31fad7f228d455d92ca3e6ab12458f9/users/47db4ef4868e446890f9e5394ef6fbb1/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:54.762198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949844ef-6c33-4c11-9fb5-9e64151fdb67 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.762448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949844ef-6c33-4c11-9fb5-9e64151fdb67 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.762810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949844ef-6c33-4c11-9fb5-9e64151fdb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.769977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:54.770924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683b27a-0816-4319-a4c8-ad43c4f2b3e0 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.795078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-949844ef-6c33-4c11-9fb5-9e64151fdb67 None None] MFA Rules not processed for user `47db4ef4868e446890f9e5394ef6fbb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:54.796684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-949844ef-6c33-4c11-9fb5-9e64151fdb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:54.850851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3e02-4423-43f8-b382-4c4593ba6590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:54.851646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3e02-4423-43f8-b382-4c4593ba6590 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:54.851923 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3e02-4423-43f8-b382-4c4593ba6590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:54.852214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3e02-4423-43f8-b382-4c4593ba6590 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:54.852900 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3e02-4423-43f8-b382-4c4593ba6590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:54.855000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3e02-4423-43f8-b382-4c4593ba6590 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:54.866837 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:54.888753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1045/2068] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:54.890907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1024/2069] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:54 2022] POST /identity/v3/projects => generated 357 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:54.901259 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949844ef-6c33-4c11-9fb5-9e64151fdb67 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.901425 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3e02-4423-43f8-b382-4c4593ba6590 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:54.903326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a54d3e02-4423-43f8-b382-4c4593ba6590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:54.922049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949844ef-6c33-4c11-9fb5-9e64151fdb67 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:54.997579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3e02-4423-43f8-b382-4c4593ba6590 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:55.001984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e21ed-72f9-471b-8116-9224683affb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:55.002959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e21ed-72f9-471b-8116-9224683affb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:55.003165 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e21ed-72f9-471b-8116-9224683affb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:55.003446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e21ed-72f9-471b-8116-9224683affb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:55.005646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe64111-6bde-4380-ac5f-fe29a364a79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:55.006399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe64111-6bde-4380-ac5f-fe29a364a79c admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:55.006680 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe64111-6bde-4380-ac5f-fe29a364a79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:55.006949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe64111-6bde-4380-ac5f-fe29a364a79c admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:55.007631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe64111-6bde-4380-ac5f-fe29a364a79c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:55.009902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe64111-6bde-4380-ac5f-fe29a364a79c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:55.034729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1046/2070] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:54 2022] POST /identity/v3/users => generated 285 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:55.038664 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe64111-6bde-4380-ac5f-fe29a364a79c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:55.052353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe64111-6bde-4380-ac5f-fe29a364a79c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:55.057470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb98a6-3e6d-41d5-8fb2-13e6893282b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:55.058235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb98a6-3e6d-41d5-8fb2-13e6893282b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:55.058585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb98a6-3e6d-41d5-8fb2-13e6893282b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:55.058877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb98a6-3e6d-41d5-8fb2-13e6893282b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:55.059585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb98a6-3e6d-41d5-8fb2-13e6893282b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:55.061190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb98a6-3e6d-41d5-8fb2-13e6893282b6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:55.071474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e21ed-72f9-471b-8116-9224683affb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:55.076425 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e21ed-72f9-471b-8116-9224683affb1 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:55.076583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1047/2071] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:55.077858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb98a6-3e6d-41d5-8fb2-13e6893282b6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:55.089580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb98a6-3e6d-41d5-8fb2-13e6893282b6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:55.094685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625926a2-a317-4b03-ba3f-95d24d47430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:55.095901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625926a2-a317-4b03-ba3f-95d24d47430c admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:55.096253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625926a2-a317-4b03-ba3f-95d24d47430c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:55.096720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625926a2-a317-4b03-ba3f-95d24d47430c admin admin] PATH_INFO: `/v3/projects/9d3bb45de9ce4db8bd704eb5efb90795/users/fd4e0272eacf4293afd562da3bfeea08/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:55.096920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1025/2072] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:55.112660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625926a2-a317-4b03-ba3f-95d24d47430c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3bb45de9ce4db8bd704eb5efb90795, user_id=fd4e0272eacf4293afd562da3bfeea08, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:55.115390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625926a2-a317-4b03-ba3f-95d24d47430c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:55.130411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1048/2073] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:55 2022] PUT /identity/v3/projects/9d3bb45de9ce4db8bd704eb5efb90795/users/fd4e0272eacf4293afd562da3bfeea08/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:55.134226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e21ed-72f9-471b-8116-9224683affb1 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:55.146417 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e21ed-72f9-471b-8116-9224683affb1 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:55.218263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee273a-b573-4ffb-957c-244ffd130c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:55.219091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee273a-b573-4ffb-957c-244ffd130c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:55.219707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee273a-b573-4ffb-957c-244ffd130c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:55.219945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee273a-b573-4ffb-957c-244ffd130c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:55.222936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee273a-b573-4ffb-957c-244ffd130c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:55.224594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee273a-b573-4ffb-957c-244ffd130c7b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:55.235228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1026/2074] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:55.240543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625926a2-a317-4b03-ba3f-95d24d47430c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:55.251374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625926a2-a317-4b03-ba3f-95d24d47430c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:55.256526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e2f98-3312-4403-9afa-f50a1231ec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:55.257471 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e2f98-3312-4403-9afa-f50a1231ec75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:55.257692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e2f98-3312-4403-9afa-f50a1231ec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:55.258004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e2f98-3312-4403-9afa-f50a1231ec75 admin admin] PATH_INFO: `/v3/projects/9d3bb45de9ce4db8bd704eb5efb90795/users/fd4e0272eacf4293afd562da3bfeea08/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:55.261410 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e2f98-3312-4403-9afa-f50a1231ec75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3bb45de9ce4db8bd704eb5efb90795, user_id=fd4e0272eacf4293afd562da3bfeea08, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:55.264391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e2f98-3312-4403-9afa-f50a1231ec75 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:55.287740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1049/2075] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:55 2022] PUT /identity/v3/projects/9d3bb45de9ce4db8bd704eb5efb90795/users/fd4e0272eacf4293afd562da3bfeea08/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:55.298460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682db4e6-9df5-48fc-95f9-7564e8fb6326 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:55.298872 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682db4e6-9df5-48fc-95f9-7564e8fb6326 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:55.299153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682db4e6-9df5-48fc-95f9-7564e8fb6326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:55.343341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-682db4e6-9df5-48fc-95f9-7564e8fb6326 None None] MFA Rules not processed for user `fd4e0272eacf4293afd562da3bfeea08`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:55.344753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-682db4e6-9df5-48fc-95f9-7564e8fb6326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:55.388982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:55.473007 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1027/2076] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:55 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:55.489071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e2f98-3312-4403-9afa-f50a1231ec75 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:55.491343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5e9e2f98-3312-4403-9afa-f50a1231ec75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:55.543471 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:55.567256 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e2f98-3312-4403-9afa-f50a1231ec75 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:55.571601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc656bac-2b24-478d-9b78-3cf23ca8d27c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:55.572337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc656bac-2b24-478d-9b78-3cf23ca8d27c service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:55.573070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc656bac-2b24-478d-9b78-3cf23ca8d27c service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:55.573566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc656bac-2b24-478d-9b78-3cf23ca8d27c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:55.592662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc656bac-2b24-478d-9b78-3cf23ca8d27c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:55.594308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc656bac-2b24-478d-9b78-3cf23ca8d27c service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:55.605942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1050/2077] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:55 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:18:55.691218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682db4e6-9df5-48fc-95f9-7564e8fb6326 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:55.700420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682db4e6-9df5-48fc-95f9-7564e8fb6326 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:55.704036 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbae2b-a122-4261-a8f2-2899ac012a7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:55.705438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbae2b-a122-4261-a8f2-2899ac012a7f service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:55.705982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbae2b-a122-4261-a8f2-2899ac012a7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:55.706310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbae2b-a122-4261-a8f2-2899ac012a7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:55.765478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbae2b-a122-4261-a8f2-2899ac012a7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:55.767633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbae2b-a122-4261-a8f2-2899ac012a7f service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:55.776885 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1028/2078] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:55 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:55.947926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc656bac-2b24-478d-9b78-3cf23ca8d27c service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:55.949362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cc656bac-2b24-478d-9b78-3cf23ca8d27c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:56.021656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc656bac-2b24-478d-9b78-3cf23ca8d27c service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:56.026265 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133309f3-b9bb-4f53-9443-f29c0e8590fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:56.027127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133309f3-b9bb-4f53-9443-f29c0e8590fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:56.027510 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133309f3-b9bb-4f53-9443-f29c0e8590fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:56.027667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133309f3-b9bb-4f53-9443-f29c0e8590fa admin admin] PATH_INFO: `/v3/users/02438387c012405eb258a65311c092a5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:56.028839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133309f3-b9bb-4f53-9443-f29c0e8590fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=02438387c012405eb258a65311c092a5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:56.030815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133309f3-b9bb-4f53-9443-f29c0e8590fa admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:56.076915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-133309f3-b9bb-4f53-9443-f29c0e8590fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02438387c012405eb258a65311c092a5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:56.084200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-133309f3-b9bb-4f53-9443-f29c0e8590fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02438387c012405eb258a65311c092a5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:56.084801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-133309f3-b9bb-4f53-9443-f29c0e8590fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02438387c012405eb258a65311c092a5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:56.085397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-133309f3-b9bb-4f53-9443-f29c0e8590fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02438387c012405eb258a65311c092a5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:56.097621 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-133309f3-b9bb-4f53-9443-f29c0e8590fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02438387c012405eb258a65311c092a5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:56.109809 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1051/2079] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:18:55 2022] DELETE /identity/v3/users/02438387c012405eb258a65311c092a5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:56.327194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbae2b-a122-4261-a8f2-2899ac012a7f service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:56.338674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbae2b-a122-4261-a8f2-2899ac012a7f service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:56.414404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748940fd-e606-45fe-aa23-d10260861df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:56.415181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748940fd-e606-45fe-aa23-d10260861df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:56.415464 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748940fd-e606-45fe-aa23-d10260861df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:56.415734 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748940fd-e606-45fe-aa23-d10260861df3 admin admin] PATH_INFO: `/v3/projects/3f65ddb707cf4a3991b4797726d1336e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:56.416898 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748940fd-e606-45fe-aa23-d10260861df3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f65ddb707cf4a3991b4797726d1336e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:56.418877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748940fd-e606-45fe-aa23-d10260861df3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:56.466000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-748940fd-e606-45fe-aa23-d10260861df3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f65ddb707cf4a3991b4797726d1336e', 'request_id': 'req-748940fd-e606-45fe-aa23-d10260861df3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:56.475212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-748940fd-e606-45fe-aa23-d10260861df3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f65ddb707cf4a3991b4797726d1336e', 'request_id': 'req-748940fd-e606-45fe-aa23-d10260861df3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:56.482865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-748940fd-e606-45fe-aa23-d10260861df3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f65ddb707cf4a3991b4797726d1336e', 'request_id': 'req-748940fd-e606-45fe-aa23-d10260861df3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:56.489342 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:56.498904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-748940fd-e606-45fe-aa23-d10260861df3 admin admin] The token cache is being invalidate because project 3f65ddb707cf4a3991b4797726d1336e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:56.499195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-748940fd-e606-45fe-aa23-d10260861df3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:56.499847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-748940fd-e606-45fe-aa23-d10260861df3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:56.501444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1029/2080] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:18:56 2022] DELETE /identity/v3/projects/3f65ddb707cf4a3991b4797726d1336e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:56.506441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ede020-544a-4843-83a1-6a7e8bcf062a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:56.506734 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ede020-544a-4843-83a1-6a7e8bcf062a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:56.507016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ede020-544a-4843-83a1-6a7e8bcf062a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:56.525323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-57ede020-544a-4843-83a1-6a7e8bcf062a None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:56.526556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-57ede020-544a-4843-83a1-6a7e8bcf062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:56.576392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:56.598190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1052/2081] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:56.606121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c28111-ce3e-4347-b2c7-211139134fe5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:56.606409 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c28111-ce3e-4347-b2c7-211139134fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:56.606713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c28111-ce3e-4347-b2c7-211139134fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:56.625661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e2c28111-ce3e-4347-b2c7-211139134fe5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:56.626975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e2c28111-ce3e-4347-b2c7-211139134fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:56.635024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1030/2082] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:56.639240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ede020-544a-4843-83a1-6a7e8bcf062a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:56.649308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ede020-544a-4843-83a1-6a7e8bcf062a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:56.653991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568262a-e439-4987-bf0c-0b7995f9506c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:56.654667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568262a-e439-4987-bf0c-0b7995f9506c admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:56.654950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568262a-e439-4987-bf0c-0b7995f9506c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:56.655218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568262a-e439-4987-bf0c-0b7995f9506c admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:56.655948 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568262a-e439-4987-bf0c-0b7995f9506c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:56.656252 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568262a-e439-4987-bf0c-0b7995f9506c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:56.658417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568262a-e439-4987-bf0c-0b7995f9506c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:56.670572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1053/2083] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:56.676248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c28111-ce3e-4347-b2c7-211139134fe5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:56.685408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c28111-ce3e-4347-b2c7-211139134fe5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:56.690387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b94112-a472-46d4-ad30-3779d7e04755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:56.691035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b94112-a472-46d4-ad30-3779d7e04755 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:56.691466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b94112-a472-46d4-ad30-3779d7e04755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:56.691741 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b94112-a472-46d4-ad30-3779d7e04755 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:56.692449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b94112-a472-46d4-ad30-3779d7e04755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:56.694408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b94112-a472-46d4-ad30-3779d7e04755 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:56.722776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1031/2084] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:18:56 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:56.726224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568262a-e439-4987-bf0c-0b7995f9506c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:56.736948 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568262a-e439-4987-bf0c-0b7995f9506c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:56.792139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8c97a-1a63-4dce-ae56-4d245b74cee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:56.792792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8c97a-1a63-4dce-ae56-4d245b74cee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:56.793068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8c97a-1a63-4dce-ae56-4d245b74cee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:56.793345 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8c97a-1a63-4dce-ae56-4d245b74cee1 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:56.794063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8c97a-1a63-4dce-ae56-4d245b74cee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:56.795977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8c97a-1a63-4dce-ae56-4d245b74cee1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:56.815739 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1054/2085] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:18:56 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:56.820128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b94112-a472-46d4-ad30-3779d7e04755 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:56.831069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b94112-a472-46d4-ad30-3779d7e04755 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:56.835640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d76eb9-5d8c-44d9-9963-135fd9be6a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:56.836290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d76eb9-5d8c-44d9-9963-135fd9be6a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:56.836574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d76eb9-5d8c-44d9-9963-135fd9be6a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:56.836850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d76eb9-5d8c-44d9-9963-135fd9be6a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:56.837595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d76eb9-5d8c-44d9-9963-135fd9be6a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:56.839261 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d76eb9-5d8c-44d9-9963-135fd9be6a18 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:56.850178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1032/2086] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:56.854007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8c97a-1a63-4dce-ae56-4d245b74cee1 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:56.864130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8c97a-1a63-4dce-ae56-4d245b74cee1 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:56.868685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35780c8c-cf25-4767-9d73-95a016a0dc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:56.869329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35780c8c-cf25-4767-9d73-95a016a0dc35 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:56.869607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35780c8c-cf25-4767-9d73-95a016a0dc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:56.869915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35780c8c-cf25-4767-9d73-95a016a0dc35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:56.870631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35780c8c-cf25-4767-9d73-95a016a0dc35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:56.872239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35780c8c-cf25-4767-9d73-95a016a0dc35 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:56.882411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1055/2087] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:56.886325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d76eb9-5d8c-44d9-9963-135fd9be6a18 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:56.897290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d76eb9-5d8c-44d9-9963-135fd9be6a18 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:56.901990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc8d46-ffb7-493b-b3a5-de642069559c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:56.902864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc8d46-ffb7-493b-b3a5-de642069559c admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:56.903147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc8d46-ffb7-493b-b3a5-de642069559c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:56.903418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc8d46-ffb7-493b-b3a5-de642069559c admin admin] PATH_INFO: `/v3/projects/84dc70884aff4a799bb8f38899766e81/users/7f2ffbc59c60455cb9cadda081e30381/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:56.919245 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc8d46-ffb7-493b-b3a5-de642069559c admin admin] RBAC: Authorizing `identity:create_grant(project_id=84dc70884aff4a799bb8f38899766e81, user_id=7f2ffbc59c60455cb9cadda081e30381, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:56.922258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc8d46-ffb7-493b-b3a5-de642069559c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:56.936521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1033/2088] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:18:56 2022] PUT /identity/v3/projects/84dc70884aff4a799bb8f38899766e81/users/7f2ffbc59c60455cb9cadda081e30381/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:56.941970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a86d5-25b3-4f09-bad2-9e6adb67e5e0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:56.942260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a86d5-25b3-4f09-bad2-9e6adb67e5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:56.942536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a86d5-25b3-4f09-bad2-9e6adb67e5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:56.960807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b20a86d5-25b3-4f09-bad2-9e6adb67e5e0 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:56.962081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b20a86d5-25b3-4f09-bad2-9e6adb67e5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:57.022850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1056/2089] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:57.032467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc8d46-ffb7-493b-b3a5-de642069559c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:57.034792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-94bc8d46-ffb7-493b-b3a5-de642069559c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:57.093113 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc8d46-ffb7-493b-b3a5-de642069559c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:57.097592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1cdb9-abbc-4b1a-8925-ca9e2f2290aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:57.098261 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1cdb9-abbc-4b1a-8925-ca9e2f2290aa service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:57.098539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1cdb9-abbc-4b1a-8925-ca9e2f2290aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:57.098811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1cdb9-abbc-4b1a-8925-ca9e2f2290aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:57.107698 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1cdb9-abbc-4b1a-8925-ca9e2f2290aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:57.109159 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1cdb9-abbc-4b1a-8925-ca9e2f2290aa service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:57.118597 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1034/2090] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:02.014060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47692f53-2aca-4d35-bdb0-9586afe6b6ac None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:02.014903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47692f53-2aca-4d35-bdb0-9586afe6b6ac None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:02.015403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47692f53-2aca-4d35-bdb0-9586afe6b6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:02.023297 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1cdb9-abbc-4b1a-8925-ca9e2f2290aa service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:02.024942 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8db1cdb9-abbc-4b1a-8925-ca9e2f2290aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:02.050061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-47692f53-2aca-4d35-bdb0-9586afe6b6ac None None] MFA Rules not processed for user `7f2ffbc59c60455cb9cadda081e30381`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:02.051801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-47692f53-2aca-4d35-bdb0-9586afe6b6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:02.097182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1cdb9-abbc-4b1a-8925-ca9e2f2290aa service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:02.102902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e47b2a-cac7-429a-9667-8b6b58abac4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:02.103985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e47b2a-cac7-429a-9667-8b6b58abac4a service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:02.104544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e47b2a-cac7-429a-9667-8b6b58abac4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:02.105039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e47b2a-cac7-429a-9667-8b6b58abac4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:02.107713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f9e47b2a-cac7-429a-9667-8b6b58abac4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:02.164783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:02.173872 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:02.196974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e47b2a-cac7-429a-9667-8b6b58abac4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:02.198572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e47b2a-cac7-429a-9667-8b6b58abac4a service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:02.222286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1035/2092] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:19:02.223180 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1057/2092] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:02.239636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e47b2a-cac7-429a-9667-8b6b58abac4a service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:02.241006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f9e47b2a-cac7-429a-9667-8b6b58abac4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:02.400252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e47b2a-cac7-429a-9667-8b6b58abac4a service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:02.424921 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5bc59-d377-406b-a5af-efc9b52f6eb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:02.425511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5bc59-d377-406b-a5af-efc9b52f6eb5 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:02.425826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5bc59-d377-406b-a5af-efc9b52f6eb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:02.426122 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5bc59-d377-406b-a5af-efc9b52f6eb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:02.455411 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5bc59-d377-406b-a5af-efc9b52f6eb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:02.458739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5bc59-d377-406b-a5af-efc9b52f6eb5 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:02.472021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1036/2093] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:04.461707 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25381b-12ef-4890-8cfe-847358b95052 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:04.462237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25381b-12ef-4890-8cfe-847358b95052 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:04.462482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25381b-12ef-4890-8cfe-847358b95052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:04.497974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3d25381b-12ef-4890-8cfe-847358b95052 None None] MFA Rules not processed for user `7f2ffbc59c60455cb9cadda081e30381`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:04.497974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3d25381b-12ef-4890-8cfe-847358b95052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:04.521059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1058/2094] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:07.778472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5bc59-d377-406b-a5af-efc9b52f6eb5 service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:07.779988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e9d5bc59-d377-406b-a5af-efc9b52f6eb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:07.795767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5bc59-d377-406b-a5af-efc9b52f6eb5 service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:07.802894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:07.803886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:07.804165 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:07.804507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] PATH_INFO: `/v3/users/f419e66145de47ffad7bc23fd606434e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:07.805819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f419e66145de47ffad7bc23fd606434e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:07.808549 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:07.864805 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f419e66145de47ffad7bc23fd606434e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:07.877916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f419e66145de47ffad7bc23fd606434e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:07.893676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f419e66145de47ffad7bc23fd606434e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:07.894767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f419e66145de47ffad7bc23fd606434e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:07.905868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f419e66145de47ffad7bc23fd606434e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:07.913121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1037/2095] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:07 2022] DELETE /identity/v3/users/f419e66145de47ffad7bc23fd606434e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:07.917594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeaf88-6239-4a85-9f3b-0bcc075053c4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:07.918165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeaf88-6239-4a85-9f3b-0bcc075053c4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:07.918704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeaf88-6239-4a85-9f3b-0bcc075053c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:07.945493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-debeaf88-6239-4a85-9f3b-0bcc075053c4 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:07.947301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-debeaf88-6239-4a85-9f3b-0bcc075053c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:08.028736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1059/2096] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:08.039605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:08.041034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:08.091570 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:08.106843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ea155-e7b8-4229-8370-b1c110015709 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:08.111225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdca9b4-018b-4356-a89f-7933ab22a4b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:08.111948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdca9b4-018b-4356-a89f-7933ab22a4b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:08.112228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdca9b4-018b-4356-a89f-7933ab22a4b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:08.112498 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdca9b4-018b-4356-a89f-7933ab22a4b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:08.121864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdca9b4-018b-4356-a89f-7933ab22a4b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:08.123306 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdca9b4-018b-4356-a89f-7933ab22a4b3 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:08.134269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1038/2097] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:08.581983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeaf88-6239-4a85-9f3b-0bcc075053c4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:08.583597 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-debeaf88-6239-4a85-9f3b-0bcc075053c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:08.598692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeaf88-6239-4a85-9f3b-0bcc075053c4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:08.603546 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:08.604563 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:08.604852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:08.605259 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] PATH_INFO: `/v3/projects/b6429497772e4503836688b7a341bba4` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:08.606482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6429497772e4503836688b7a341bba4)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:08.608492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:08.681854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6429497772e4503836688b7a341bba4', 'request_id': 'req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:08.689376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6429497772e4503836688b7a341bba4', 'request_id': 'req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:08.697535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6429497772e4503836688b7a341bba4', 'request_id': 'req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:08.738405 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] The token cache is being invalidate because project b6429497772e4503836688b7a341bba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:08.738767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:08.740489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:08.743188 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1060/2098] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:08 2022] DELETE /identity/v3/projects/b6429497772e4503836688b7a341bba4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:08.747265 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdca9b4-018b-4356-a89f-7933ab22a4b3 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:08.758064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdca9b4-018b-4356-a89f-7933ab22a4b3 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:08.844673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8c148-2834-46b1-bbfd-26cc86128f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:08.846676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8c148-2834-46b1-bbfd-26cc86128f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:08.847154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8c148-2834-46b1-bbfd-26cc86128f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:08.847594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8c148-2834-46b1-bbfd-26cc86128f67 admin admin] PATH_INFO: `/v3/users/47db4ef4868e446890f9e5394ef6fbb1` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:08.849958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8c148-2834-46b1-bbfd-26cc86128f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47db4ef4868e446890f9e5394ef6fbb1)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:08.853376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8c148-2834-46b1-bbfd-26cc86128f67 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:08.917682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-14d8c148-2834-46b1-bbfd-26cc86128f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47db4ef4868e446890f9e5394ef6fbb1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:08.954876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-14d8c148-2834-46b1-bbfd-26cc86128f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47db4ef4868e446890f9e5394ef6fbb1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:08.983562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-14d8c148-2834-46b1-bbfd-26cc86128f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47db4ef4868e446890f9e5394ef6fbb1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:08.984613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-14d8c148-2834-46b1-bbfd-26cc86128f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47db4ef4868e446890f9e5394ef6fbb1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:09.006703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-14d8c148-2834-46b1-bbfd-26cc86128f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47db4ef4868e446890f9e5394ef6fbb1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:09.018909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1039/2099] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:08 2022] DELETE /identity/v3/users/47db4ef4868e446890f9e5394ef6fbb1 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:09.393687 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:09.395304 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:09.427984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538f33e-4d5e-4e0d-abb9-f59a8b29baa6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:09.532026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb7419-661d-42da-8e06-3d6878068aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:09.534456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb7419-661d-42da-8e06-3d6878068aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:09.534692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb7419-661d-42da-8e06-3d6878068aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:09.534958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb7419-661d-42da-8e06-3d6878068aa5 admin admin] PATH_INFO: `/v3/projects/f31fad7f228d455d92ca3e6ab12458f9` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:09.536162 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb7419-661d-42da-8e06-3d6878068aa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f31fad7f228d455d92ca3e6ab12458f9)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:09.538119 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb7419-661d-42da-8e06-3d6878068aa5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:09.544858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:09.588553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dfdb7419-661d-42da-8e06-3d6878068aa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f31fad7f228d455d92ca3e6ab12458f9', 'request_id': 'req-dfdb7419-661d-42da-8e06-3d6878068aa5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:09.594678 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dfdb7419-661d-42da-8e06-3d6878068aa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f31fad7f228d455d92ca3e6ab12458f9', 'request_id': 'req-dfdb7419-661d-42da-8e06-3d6878068aa5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:09.601184 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dfdb7419-661d-42da-8e06-3d6878068aa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f31fad7f228d455d92ca3e6ab12458f9', 'request_id': 'req-dfdb7419-661d-42da-8e06-3d6878068aa5'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:09.614055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dfdb7419-661d-42da-8e06-3d6878068aa5 admin admin] The token cache is being invalidate because project f31fad7f228d455d92ca3e6ab12458f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:09.614341 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dfdb7419-661d-42da-8e06-3d6878068aa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:09.614974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dfdb7419-661d-42da-8e06-3d6878068aa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:09.616569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1061/2100] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:09 2022] DELETE /identity/v3/projects/f31fad7f228d455d92ca3e6ab12458f9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:09.622547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d76ef5-0733-48bd-82c8-2b84294c993d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:09.622840 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d76ef5-0733-48bd-82c8-2b84294c993d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:09.623145 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d76ef5-0733-48bd-82c8-2b84294c993d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:09.643229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-33d76ef5-0733-48bd-82c8-2b84294c993d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:09.644499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-33d76ef5-0733-48bd-82c8-2b84294c993d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:09.708169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1040/2101] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:09.716615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b5ce5-2a62-48a9-896c-3d8df7bf7779 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:09.716902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b5ce5-2a62-48a9-896c-3d8df7bf7779 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:09.717173 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b5ce5-2a62-48a9-896c-3d8df7bf7779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:09.736370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2a8b5ce5-2a62-48a9-896c-3d8df7bf7779 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:09.737558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2a8b5ce5-2a62-48a9-896c-3d8df7bf7779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:09.745622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1062/2102] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:09.749872 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d76ef5-0733-48bd-82c8-2b84294c993d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:09.755600 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:09.762243 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d76ef5-0733-48bd-82c8-2b84294c993d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:09.769496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add3c98-5a15-42e5-acae-87653f3139ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:09.770179 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add3c98-5a15-42e5-acae-87653f3139ce admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:09.770461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add3c98-5a15-42e5-acae-87653f3139ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:09.770736 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add3c98-5a15-42e5-acae-87653f3139ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:09.771475 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add3c98-5a15-42e5-acae-87653f3139ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:19:09.771800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add3c98-5a15-42e5-acae-87653f3139ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:09.773428 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add3c98-5a15-42e5-acae-87653f3139ce admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:09.786335 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1041/2103] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:19:09.790873 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b5ce5-2a62-48a9-896c-3d8df7bf7779 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:09.801130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b5ce5-2a62-48a9-896c-3d8df7bf7779 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:09.806269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7cd57c-df70-48d3-a8ea-8a6d8291056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:09.806644 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cd57c-df70-48d3-a8ea-8a6d8291056c admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:09.806807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cd57c-df70-48d3-a8ea-8a6d8291056c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:09.807072 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cd57c-df70-48d3-a8ea-8a6d8291056c admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:09.807766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7cd57c-df70-48d3-a8ea-8a6d8291056c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:09.809721 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7cd57c-df70-48d3-a8ea-8a6d8291056c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:09.838706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1063/2104] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:09 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:09.842949 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add3c98-5a15-42e5-acae-87653f3139ce admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:09.854404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add3c98-5a15-42e5-acae-87653f3139ce admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:09.910403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86310a1e-92d1-485a-a980-06393ca016ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:09.911069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86310a1e-92d1-485a-a980-06393ca016ac admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:09.911350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86310a1e-92d1-485a-a980-06393ca016ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:09.911626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86310a1e-92d1-485a-a980-06393ca016ac admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:09.912319 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86310a1e-92d1-485a-a980-06393ca016ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:09.914269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86310a1e-92d1-485a-a980-06393ca016ac admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:09.935029 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1042/2105] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:09 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:09.940443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7cd57c-df70-48d3-a8ea-8a6d8291056c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:09.951891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7cd57c-df70-48d3-a8ea-8a6d8291056c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:09.957156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53bb14-1bd4-4e64-9a0f-2e1eb756cdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:09.957884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53bb14-1bd4-4e64-9a0f-2e1eb756cdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:09.958168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53bb14-1bd4-4e64-9a0f-2e1eb756cdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:09.958445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53bb14-1bd4-4e64-9a0f-2e1eb756cdf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:09.959165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53bb14-1bd4-4e64-9a0f-2e1eb756cdf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:09.960786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53bb14-1bd4-4e64-9a0f-2e1eb756cdf5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:09.971321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1064/2106] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:09.975569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86310a1e-92d1-485a-a980-06393ca016ac admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:09.985538 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86310a1e-92d1-485a-a980-06393ca016ac admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:09.990444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d7f19-4a2a-49ae-a147-26af04860886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:09.991088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d7f19-4a2a-49ae-a147-26af04860886 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:09.991368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d7f19-4a2a-49ae-a147-26af04860886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:09.991637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d7f19-4a2a-49ae-a147-26af04860886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:09.992348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d7f19-4a2a-49ae-a147-26af04860886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:09.993991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d7f19-4a2a-49ae-a147-26af04860886 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:10.005686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1043/2107] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:10.009969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53bb14-1bd4-4e64-9a0f-2e1eb756cdf5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:10.020729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53bb14-1bd4-4e64-9a0f-2e1eb756cdf5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:10.025406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a757a-7570-47ce-9587-f051e47c56e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:10.026314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a757a-7570-47ce-9587-f051e47c56e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:10.026597 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a757a-7570-47ce-9587-f051e47c56e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:10.026870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a757a-7570-47ce-9587-f051e47c56e6 admin admin] PATH_INFO: `/v3/projects/3c7ec7871ce949ea8993d03976d08c93/users/43bc9564e3dd4c278a14aca86f7ac951/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:10.041287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a757a-7570-47ce-9587-f051e47c56e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7ec7871ce949ea8993d03976d08c93, user_id=43bc9564e3dd4c278a14aca86f7ac951, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:10.043980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a757a-7570-47ce-9587-f051e47c56e6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:10.056713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1065/2108] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:10 2022] PUT /identity/v3/projects/3c7ec7871ce949ea8993d03976d08c93/users/43bc9564e3dd4c278a14aca86f7ac951/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:10.068707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546339bf-4533-4558-b4fa-f955deb3286d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:10.069012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546339bf-4533-4558-b4fa-f955deb3286d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:10.069375 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546339bf-4533-4558-b4fa-f955deb3286d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:10.100358 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-546339bf-4533-4558-b4fa-f955deb3286d None None] MFA Rules not processed for user `43bc9564e3dd4c278a14aca86f7ac951`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:10.101498 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-546339bf-4533-4558-b4fa-f955deb3286d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:10.164854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1044/2109] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:10.176581 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a757a-7570-47ce-9587-f051e47c56e6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:10.178318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-686a757a-7570-47ce-9587-f051e47c56e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:10.236165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a757a-7570-47ce-9587-f051e47c56e6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:10.240438 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3fcab-c4bd-4d5a-8324-e488da2f2d82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:10.241145 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3fcab-c4bd-4d5a-8324-e488da2f2d82 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:10.241429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3fcab-c4bd-4d5a-8324-e488da2f2d82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:10.241823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3fcab-c4bd-4d5a-8324-e488da2f2d82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:10.250222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3fcab-c4bd-4d5a-8324-e488da2f2d82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:10.252284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3fcab-c4bd-4d5a-8324-e488da2f2d82 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:10.262658 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1066/2110] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:10.282804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546339bf-4533-4558-b4fa-f955deb3286d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:10.354121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:10.358696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546339bf-4533-4558-b4fa-f955deb3286d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:10.366114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:10.367246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:10.367626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:10.367969 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] PATH_INFO: `/v3/users/43bc9564e3dd4c278a14aca86f7ac951` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:10.369244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=43bc9564e3dd4c278a14aca86f7ac951)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:10.371296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:10.432429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43bc9564e3dd4c278a14aca86f7ac951'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:10.441801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43bc9564e3dd4c278a14aca86f7ac951'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:10.457037 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43bc9564e3dd4c278a14aca86f7ac951'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:10.457855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43bc9564e3dd4c278a14aca86f7ac951'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:10.465960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43bc9564e3dd4c278a14aca86f7ac951'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:10.473765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1045/2111] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:10 2022] DELETE /identity/v3/users/43bc9564e3dd4c278a14aca86f7ac951 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:10.478056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1644-34db-407b-84a5-866b22ddcf33 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:10.478524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1644-34db-407b-84a5-866b22ddcf33 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:10.478749 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1644-34db-407b-84a5-866b22ddcf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:10.498177 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a14a1644-34db-407b-84a5-866b22ddcf33 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:10.499402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a14a1644-34db-407b-84a5-866b22ddcf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:10.541137 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:10.577447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1067/2112] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:10.589441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:10.590914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:10.653520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0554b60-e05e-401b-ba57-1ee122cd5bfb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:10.659631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae08f8-333f-4313-a918-e8051d1f25c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:10.660313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae08f8-333f-4313-a918-e8051d1f25c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:10.660586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae08f8-333f-4313-a918-e8051d1f25c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:10.660859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae08f8-333f-4313-a918-e8051d1f25c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:10.671290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae08f8-333f-4313-a918-e8051d1f25c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:10.673600 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae08f8-333f-4313-a918-e8051d1f25c7 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:10.684639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1046/2113] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:10.987924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1644-34db-407b-84a5-866b22ddcf33 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:10.989894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a14a1644-34db-407b-84a5-866b22ddcf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:11.006445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1644-34db-407b-84a5-866b22ddcf33 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:11.011710 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52900d-431d-491c-ab9a-5f47f3899991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:11.012784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52900d-431d-491c-ab9a-5f47f3899991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.013141 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52900d-431d-491c-ab9a-5f47f3899991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.013511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52900d-431d-491c-ab9a-5f47f3899991 admin admin] PATH_INFO: `/v3/projects/3c7ec7871ce949ea8993d03976d08c93` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.016341 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52900d-431d-491c-ab9a-5f47f3899991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7ec7871ce949ea8993d03976d08c93)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:11.018961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52900d-431d-491c-ab9a-5f47f3899991 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:11.077364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ef52900d-431d-491c-ab9a-5f47f3899991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7ec7871ce949ea8993d03976d08c93', 'request_id': 'req-ef52900d-431d-491c-ab9a-5f47f3899991'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:11.086625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ef52900d-431d-491c-ab9a-5f47f3899991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7ec7871ce949ea8993d03976d08c93', 'request_id': 'req-ef52900d-431d-491c-ab9a-5f47f3899991'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:11.093074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ef52900d-431d-491c-ab9a-5f47f3899991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7ec7871ce949ea8993d03976d08c93', 'request_id': 'req-ef52900d-431d-491c-ab9a-5f47f3899991'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:11.106522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ef52900d-431d-491c-ab9a-5f47f3899991 admin admin] The token cache is being invalidate because project 3c7ec7871ce949ea8993d03976d08c93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:11.106810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ef52900d-431d-491c-ab9a-5f47f3899991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:11.107433 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ef52900d-431d-491c-ab9a-5f47f3899991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:11.108981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1068/2114] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:10 2022] DELETE /identity/v3/projects/3c7ec7871ce949ea8993d03976d08c93 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:11.114745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7f3e5-845a-4ac6-b33c-4dce07aea2dc None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.115315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7f3e5-845a-4ac6-b33c-4dce07aea2dc None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.115467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7f3e5-845a-4ac6-b33c-4dce07aea2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.138461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bcc7f3e5-845a-4ac6-b33c-4dce07aea2dc None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:11.139675 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bcc7f3e5-845a-4ac6-b33c-4dce07aea2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:11.208291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1047/2115] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:11.216476 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fd534-284c-4401-93a0-8301847cfdad None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.216963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fd534-284c-4401-93a0-8301847cfdad None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.217245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fd534-284c-4401-93a0-8301847cfdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.236132 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-235fd534-284c-4401-93a0-8301847cfdad None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:11.237320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-235fd534-284c-4401-93a0-8301847cfdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:11.245154 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1069/2116] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:11.249330 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7f3e5-845a-4ac6-b33c-4dce07aea2dc None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:11.260218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7f3e5-845a-4ac6-b33c-4dce07aea2dc None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:11.264831 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e5ba0-6513-46d6-af8e-69765ce996cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:11.265487 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e5ba0-6513-46d6-af8e-69765ce996cf admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.265952 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e5ba0-6513-46d6-af8e-69765ce996cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.266225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e5ba0-6513-46d6-af8e-69765ce996cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.267315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e5ba0-6513-46d6-af8e-69765ce996cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:19:11.267430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e5ba0-6513-46d6-af8e-69765ce996cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:11.268941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e5ba0-6513-46d6-af8e-69765ce996cf admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:11.281509 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1048/2117] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:19:11.286091 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fd534-284c-4401-93a0-8301847cfdad None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:11.295407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fd534-284c-4401-93a0-8301847cfdad None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:11.301102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460a815-3174-4314-9287-44ec9b6e2251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:11.301776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460a815-3174-4314-9287-44ec9b6e2251 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.302064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460a815-3174-4314-9287-44ec9b6e2251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.302339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460a815-3174-4314-9287-44ec9b6e2251 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.303030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460a815-3174-4314-9287-44ec9b6e2251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:11.304953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460a815-3174-4314-9287-44ec9b6e2251 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:11.331229 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1070/2118] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:11 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:11.336947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e5ba0-6513-46d6-af8e-69765ce996cf admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:11.350877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e5ba0-6513-46d6-af8e-69765ce996cf admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:11.354694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:11.410682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbe824-0ffd-4ae1-8365-0193611bec63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:11.411357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbe824-0ffd-4ae1-8365-0193611bec63 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.411636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbe824-0ffd-4ae1-8365-0193611bec63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.411908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbe824-0ffd-4ae1-8365-0193611bec63 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.412593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbe824-0ffd-4ae1-8365-0193611bec63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:11.414543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbe824-0ffd-4ae1-8365-0193611bec63 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:11.441956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1049/2119] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:11 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:11.443346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460a815-3174-4314-9287-44ec9b6e2251 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:11.454336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460a815-3174-4314-9287-44ec9b6e2251 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:11.459055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343bf95-1b44-4f89-838a-2294d063ee0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:11.459730 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343bf95-1b44-4f89-838a-2294d063ee0c admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.460007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343bf95-1b44-4f89-838a-2294d063ee0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.460276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343bf95-1b44-4f89-838a-2294d063ee0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.461001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343bf95-1b44-4f89-838a-2294d063ee0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:11.462668 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343bf95-1b44-4f89-838a-2294d063ee0c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:11.478466 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1071/2120] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:11.479611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbe824-0ffd-4ae1-8365-0193611bec63 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:11.490734 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbe824-0ffd-4ae1-8365-0193611bec63 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:11.496121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164bf51-ca6b-487b-bd3a-d78a7aaf6625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:11.497514 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164bf51-ca6b-487b-bd3a-d78a7aaf6625 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.498377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164bf51-ca6b-487b-bd3a-d78a7aaf6625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.498377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164bf51-ca6b-487b-bd3a-d78a7aaf6625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.501351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164bf51-ca6b-487b-bd3a-d78a7aaf6625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:11.503052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164bf51-ca6b-487b-bd3a-d78a7aaf6625 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:11.515007 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1050/2121] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:11.519770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343bf95-1b44-4f89-838a-2294d063ee0c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:11.530388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343bf95-1b44-4f89-838a-2294d063ee0c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:11.535854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931563f-258b-401d-8312-bad218ac889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:11.536741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931563f-258b-401d-8312-bad218ac889b admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.537019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931563f-258b-401d-8312-bad218ac889b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.537304 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931563f-258b-401d-8312-bad218ac889b admin admin] PATH_INFO: `/v3/projects/19e2a24160404eb9b71a359abc538be8/users/0623d2f89d4b42c8968fb29c3121dbae/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.554016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931563f-258b-401d-8312-bad218ac889b admin admin] RBAC: Authorizing `identity:create_grant(project_id=19e2a24160404eb9b71a359abc538be8, user_id=0623d2f89d4b42c8968fb29c3121dbae, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:11.557292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931563f-258b-401d-8312-bad218ac889b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:11.573532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1072/2122] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:11 2022] PUT /identity/v3/projects/19e2a24160404eb9b71a359abc538be8/users/0623d2f89d4b42c8968fb29c3121dbae/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:11.585062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fb496-237f-4ee9-a961-7393dbfda5fa None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.585376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fb496-237f-4ee9-a961-7393dbfda5fa None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.585621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fb496-237f-4ee9-a961-7393dbfda5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.627445 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-498fb496-237f-4ee9-a961-7393dbfda5fa None None] MFA Rules not processed for user `0623d2f89d4b42c8968fb29c3121dbae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:11.628659 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-498fb496-237f-4ee9-a961-7393dbfda5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:11.703888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1051/2123] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:11.708836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931563f-258b-401d-8312-bad218ac889b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:11.780670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:11.793295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931563f-258b-401d-8312-bad218ac889b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:11.800563 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ee586-665c-45ef-aa37-a3cedd90fb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:11.801468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ee586-665c-45ef-aa37-a3cedd90fb90 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.802012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ee586-665c-45ef-aa37-a3cedd90fb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.802358 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ee586-665c-45ef-aa37-a3cedd90fb90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.803221 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ee586-665c-45ef-aa37-a3cedd90fb90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:11.809807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ee586-665c-45ef-aa37-a3cedd90fb90 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:11.836650 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1073/2124] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:11 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:11.840677 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fb496-237f-4ee9-a961-7393dbfda5fa None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:11.853243 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fb496-237f-4ee9-a961-7393dbfda5fa None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:11.918695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c60ca1-99c3-4b94-9221-b2b02ac5eac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:11.919314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c60ca1-99c3-4b94-9221-b2b02ac5eac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.919590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c60ca1-99c3-4b94-9221-b2b02ac5eac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.919864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c60ca1-99c3-4b94-9221-b2b02ac5eac1 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.920559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c60ca1-99c3-4b94-9221-b2b02ac5eac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:11.922775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c60ca1-99c3-4b94-9221-b2b02ac5eac1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:11.943195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1052/2125] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:11 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:11.947624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ee586-665c-45ef-aa37-a3cedd90fb90 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:11.958874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ee586-665c-45ef-aa37-a3cedd90fb90 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:11.963579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf71cb8-1110-4259-b563-2ba18829e0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:11.964274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf71cb8-1110-4259-b563-2ba18829e0de admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:11.964485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf71cb8-1110-4259-b563-2ba18829e0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:11.964766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf71cb8-1110-4259-b563-2ba18829e0de admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:11.965493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf71cb8-1110-4259-b563-2ba18829e0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:11.967133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf71cb8-1110-4259-b563-2ba18829e0de admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:11.981264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1074/2126] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:11.987900 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c60ca1-99c3-4b94-9221-b2b02ac5eac1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:11.995575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:11.999500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c60ca1-99c3-4b94-9221-b2b02ac5eac1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:12.005544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d79f6-7d29-449e-8ec7-3b1dcf29abc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:12.006456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d79f6-7d29-449e-8ec7-3b1dcf29abc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:12.006736 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d79f6-7d29-449e-8ec7-3b1dcf29abc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:12.007038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d79f6-7d29-449e-8ec7-3b1dcf29abc1 admin admin] PATH_INFO: `/v3/projects/4b65381065474a2aa42e9ce620816ef3/users/19f4612c516c491ca819848be61fa070/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:12.023813 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d79f6-7d29-449e-8ec7-3b1dcf29abc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b65381065474a2aa42e9ce620816ef3, user_id=19f4612c516c491ca819848be61fa070, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:12.026635 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d79f6-7d29-449e-8ec7-3b1dcf29abc1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:12.040460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1053/2127] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:11 2022] PUT /identity/v3/projects/4b65381065474a2aa42e9ce620816ef3/users/19f4612c516c491ca819848be61fa070/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:12.044479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf71cb8-1110-4259-b563-2ba18829e0de admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:12.055396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf71cb8-1110-4259-b563-2ba18829e0de admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:12.115738 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728b6dd-b957-4513-a168-e961c88e62a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:12.116401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728b6dd-b957-4513-a168-e961c88e62a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:12.116714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728b6dd-b957-4513-a168-e961c88e62a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:12.116989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728b6dd-b957-4513-a168-e961c88e62a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:12.117712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728b6dd-b957-4513-a168-e961c88e62a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:12.119398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728b6dd-b957-4513-a168-e961c88e62a3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:12.130264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1075/2128] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:12.134445 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d79f6-7d29-449e-8ec7-3b1dcf29abc1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:12.145117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d79f6-7d29-449e-8ec7-3b1dcf29abc1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:12.150025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cdc1f-1243-4dc9-811f-bc83fb602fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:12.150916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cdc1f-1243-4dc9-811f-bc83fb602fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:12.151191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cdc1f-1243-4dc9-811f-bc83fb602fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:12.151491 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cdc1f-1243-4dc9-811f-bc83fb602fb4 admin admin] PATH_INFO: `/v3/projects/4b65381065474a2aa42e9ce620816ef3/users/19f4612c516c491ca819848be61fa070/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:12.153439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cdc1f-1243-4dc9-811f-bc83fb602fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b65381065474a2aa42e9ce620816ef3, user_id=19f4612c516c491ca819848be61fa070, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:12.156104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cdc1f-1243-4dc9-811f-bc83fb602fb4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:12.167605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1054/2129] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:12 2022] PUT /identity/v3/projects/4b65381065474a2aa42e9ce620816ef3/users/19f4612c516c491ca819848be61fa070/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:12.179024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71059a-d92e-49ed-8b1f-f32d9fe4c0f9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:12.179244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71059a-d92e-49ed-8b1f-f32d9fe4c0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:12.179518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71059a-d92e-49ed-8b1f-f32d9fe4c0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:12.205378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8f71059a-d92e-49ed-8b1f-f32d9fe4c0f9 None None] MFA Rules not processed for user `19f4612c516c491ca819848be61fa070`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:12.206759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8f71059a-d92e-49ed-8b1f-f32d9fe4c0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:12.277052 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1076/2130] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:12.288824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cdc1f-1243-4dc9-811f-bc83fb602fb4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:12.290576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-903cdc1f-1243-4dc9-811f-bc83fb602fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:12.350236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cdc1f-1243-4dc9-811f-bc83fb602fb4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:12.353907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce41d7b-d3d5-4600-875d-7efb2d6d71e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:12.354552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce41d7b-d3d5-4600-875d-7efb2d6d71e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:12.354836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce41d7b-d3d5-4600-875d-7efb2d6d71e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:12.355112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce41d7b-d3d5-4600-875d-7efb2d6d71e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:12.411836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce41d7b-d3d5-4600-875d-7efb2d6d71e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:12.413260 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce41d7b-d3d5-4600-875d-7efb2d6d71e1 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:12.422268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1055/2131] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:14.552364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71059a-d92e-49ed-8b1f-f32d9fe4c0f9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:14.553989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8f71059a-d92e-49ed-8b1f-f32d9fe4c0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:14.605286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:14.622534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71059a-d92e-49ed-8b1f-f32d9fe4c0f9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:14.627149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:14.628007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:14.628288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:14.628560 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] PATH_INFO: `/v3/users/d35af6eee793430d9d3b46599f807c17` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:14.630034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35af6eee793430d9d3b46599f807c17)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:14.631964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:14.681289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35af6eee793430d9d3b46599f807c17'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:14.688510 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35af6eee793430d9d3b46599f807c17'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:14.689412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35af6eee793430d9d3b46599f807c17'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:14.690037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35af6eee793430d9d3b46599f807c17'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:14.701547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35af6eee793430d9d3b46599f807c17'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:14.715255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1077/2132] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:14 2022] DELETE /identity/v3/users/d35af6eee793430d9d3b46599f807c17 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:14.720165 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db2eaa-7b9c-4cad-b560-29316cdc3724 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:14.720655 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db2eaa-7b9c-4cad-b560-29316cdc3724 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:14.720783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db2eaa-7b9c-4cad-b560-29316cdc3724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:14.738694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b2db2eaa-7b9c-4cad-b560-29316cdc3724 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:14.739919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b2db2eaa-7b9c-4cad-b560-29316cdc3724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:14.805153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1056/2133] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:14.815414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:14.816811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:14.879876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9d8fd-39a0-48d9-9785-df6ba14e934f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:14.883724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d4f25-f99f-46e8-a4fb-cce120ec433b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:14.884478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d4f25-f99f-46e8-a4fb-cce120ec433b service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:14.884828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d4f25-f99f-46e8-a4fb-cce120ec433b service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:14.885171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d4f25-f99f-46e8-a4fb-cce120ec433b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:14.895281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d4f25-f99f-46e8-a4fb-cce120ec433b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:14.896802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d4f25-f99f-46e8-a4fb-cce120ec433b service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:14.906490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1078/2134] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:19:15.005770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db2eaa-7b9c-4cad-b560-29316cdc3724 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:15.007184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b2db2eaa-7b9c-4cad-b560-29316cdc3724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:15.019278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db2eaa-7b9c-4cad-b560-29316cdc3724 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:15.024059 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:15.024907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:15.025181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:15.025479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] PATH_INFO: `/v3/users/0623d2f89d4b42c8968fb29c3121dbae` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:15.026713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0623d2f89d4b42c8968fb29c3121dbae)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:15.028612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:15.058249 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:15.134300 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0623d2f89d4b42c8968fb29c3121dbae'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.146592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0623d2f89d4b42c8968fb29c3121dbae'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.174379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0623d2f89d4b42c8968fb29c3121dbae'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.175813 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0623d2f89d4b42c8968fb29c3121dbae'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.190825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0623d2f89d4b42c8968fb29c3121dbae'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.199311 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1057/2135] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:15 2022] DELETE /identity/v3/users/0623d2f89d4b42c8968fb29c3121dbae => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:15.204716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37effbf-26e0-488e-aa8c-aba2e1e7613d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:15.205025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37effbf-26e0-488e-aa8c-aba2e1e7613d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:15.205343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37effbf-26e0-488e-aa8c-aba2e1e7613d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:15.224583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f37effbf-26e0-488e-aa8c-aba2e1e7613d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:15.226124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f37effbf-26e0-488e-aa8c-aba2e1e7613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:15.300219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1079/2136] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:15.309676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:15.311574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:15.396772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37effbf-26e0-488e-aa8c-aba2e1e7613d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:15.399268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f37effbf-26e0-488e-aa8c-aba2e1e7613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:15.406869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c844ec-dffa-4056-89ca-ad1d43896f89 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:15.411817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea7d37-cf29-4325-9e9b-a02ca25038da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:15.413108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea7d37-cf29-4325-9e9b-a02ca25038da service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:15.414874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea7d37-cf29-4325-9e9b-a02ca25038da service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:15.415175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea7d37-cf29-4325-9e9b-a02ca25038da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:15.417170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37effbf-26e0-488e-aa8c-aba2e1e7613d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:15.425369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:15.426605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:15.427138 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:15.427648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] PATH_INFO: `/v3/projects/dc08a57d53904aae84bfd5df87948cee` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:15.429309 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc08a57d53904aae84bfd5df87948cee)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:15.432208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:15.436002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea7d37-cf29-4325-9e9b-a02ca25038da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:15.438005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea7d37-cf29-4325-9e9b-a02ca25038da service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:15.442642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:15.457054 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1058/2137] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:19:15.499063 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc08a57d53904aae84bfd5df87948cee', 'request_id': 'req-546cd993-1532-4f27-8c74-87980055ace2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.510843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc08a57d53904aae84bfd5df87948cee', 'request_id': 'req-546cd993-1532-4f27-8c74-87980055ace2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.518538 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc08a57d53904aae84bfd5df87948cee', 'request_id': 'req-546cd993-1532-4f27-8c74-87980055ace2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.531470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] The token cache is being invalidate because project dc08a57d53904aae84bfd5df87948cee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:15.531708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.532562 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.534771 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1080/2138] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:15 2022] DELETE /identity/v3/projects/dc08a57d53904aae84bfd5df87948cee => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:15.538919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea7d37-cf29-4325-9e9b-a02ca25038da service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:15.551683 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea7d37-cf29-4325-9e9b-a02ca25038da service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:15.615226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:15.616337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:15.616617 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:15.617064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] PATH_INFO: `/v3/users/fd4e0272eacf4293afd562da3bfeea08` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:15.618605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4e0272eacf4293afd562da3bfeea08)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:15.620786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:15.666789 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:15.680636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4e0272eacf4293afd562da3bfeea08'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.689784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4e0272eacf4293afd562da3bfeea08'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.701989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4e0272eacf4293afd562da3bfeea08'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.702685 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4e0272eacf4293afd562da3bfeea08'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.710352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4e0272eacf4293afd562da3bfeea08'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.716643 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1059/2139] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:15 2022] DELETE /identity/v3/users/fd4e0272eacf4293afd562da3bfeea08 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:15.793003 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:15.794759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:15.808890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cd993-1532-4f27-8c74-87980055ace2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:15.883985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:15.884841 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:15.885117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:15.885385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] PATH_INFO: `/v3/projects/19e2a24160404eb9b71a359abc538be8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:15.886680 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19e2a24160404eb9b71a359abc538be8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:15.888589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:15.944473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19e2a24160404eb9b71a359abc538be8', 'request_id': 'req-d4b8822e-a7a9-476f-9646-d48ec7336f74'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.950858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19e2a24160404eb9b71a359abc538be8', 'request_id': 'req-d4b8822e-a7a9-476f-9646-d48ec7336f74'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.963231 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19e2a24160404eb9b71a359abc538be8', 'request_id': 'req-d4b8822e-a7a9-476f-9646-d48ec7336f74'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.977981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] The token cache is being invalidate because project 19e2a24160404eb9b71a359abc538be8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:15.978396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.979018 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:15.980611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1081/2140] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:15 2022] DELETE /identity/v3/projects/19e2a24160404eb9b71a359abc538be8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:15.984796 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:15.986425 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:15.999595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c397a-7f90-453a-ad48-99e28760674b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:16.054055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:16.055660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:16.077778 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa6d12-0ce5-4e23-9acf-0d6f0f9950c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:16.078853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa6d12-0ce5-4e23-9acf-0d6f0f9950c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.079804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa6d12-0ce5-4e23-9acf-0d6f0f9950c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.080354 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa6d12-0ce5-4e23-9acf-0d6f0f9950c4 admin admin] PATH_INFO: `/v3/users/19f4612c516c491ca819848be61fa070` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.081453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa6d12-0ce5-4e23-9acf-0d6f0f9950c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19f4612c516c491ca819848be61fa070)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:16.083676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa6d12-0ce5-4e23-9acf-0d6f0f9950c4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:16.093598 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:16.141375 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-41aa6d12-0ce5-4e23-9acf-0d6f0f9950c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19f4612c516c491ca819848be61fa070'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.143492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8822e-a7a9-476f-9646-d48ec7336f74 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:16.150112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f104ee1-ecda-44f7-a406-e3fbf144b99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:16.151113 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f104ee1-ecda-44f7-a406-e3fbf144b99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.151363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f104ee1-ecda-44f7-a406-e3fbf144b99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.151654 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f104ee1-ecda-44f7-a406-e3fbf144b99d admin admin] PATH_INFO: `/v3/projects/9d3bb45de9ce4db8bd704eb5efb90795` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.152941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f104ee1-ecda-44f7-a406-e3fbf144b99d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d3bb45de9ce4db8bd704eb5efb90795)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:16.154171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-41aa6d12-0ce5-4e23-9acf-0d6f0f9950c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19f4612c516c491ca819848be61fa070'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.154994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f104ee1-ecda-44f7-a406-e3fbf144b99d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:16.173051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-41aa6d12-0ce5-4e23-9acf-0d6f0f9950c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19f4612c516c491ca819848be61fa070'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.174267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-41aa6d12-0ce5-4e23-9acf-0d6f0f9950c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19f4612c516c491ca819848be61fa070'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.191048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-41aa6d12-0ce5-4e23-9acf-0d6f0f9950c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19f4612c516c491ca819848be61fa070'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.211083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1060/2141] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:15 2022] DELETE /identity/v3/users/19f4612c516c491ca819848be61fa070 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:16.222655 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6f104ee1-ecda-44f7-a406-e3fbf144b99d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d3bb45de9ce4db8bd704eb5efb90795', 'request_id': 'req-6f104ee1-ecda-44f7-a406-e3fbf144b99d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.229554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6f104ee1-ecda-44f7-a406-e3fbf144b99d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d3bb45de9ce4db8bd704eb5efb90795', 'request_id': 'req-6f104ee1-ecda-44f7-a406-e3fbf144b99d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.235349 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6f104ee1-ecda-44f7-a406-e3fbf144b99d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d3bb45de9ce4db8bd704eb5efb90795', 'request_id': 'req-6f104ee1-ecda-44f7-a406-e3fbf144b99d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.249464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6f104ee1-ecda-44f7-a406-e3fbf144b99d admin admin] The token cache is being invalidate because project 9d3bb45de9ce4db8bd704eb5efb90795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:16.249782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6f104ee1-ecda-44f7-a406-e3fbf144b99d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.250431 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6f104ee1-ecda-44f7-a406-e3fbf144b99d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.252959 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1082/2142] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:16 2022] DELETE /identity/v3/projects/9d3bb45de9ce4db8bd704eb5efb90795 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:16.257806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9474a-2769-4cb3-b383-573b99c4b418 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.258101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9474a-2769-4cb3-b383-573b99c4b418 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.258379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9474a-2769-4cb3-b383-573b99c4b418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.282360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-94a9474a-2769-4cb3-b383-573b99c4b418 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:16.283806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-94a9474a-2769-4cb3-b383-573b99c4b418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:16.323024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:16.358035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1061/2143] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:16.367627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e474fe-7def-4e31-a9ab-8e8811289a70 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.367997 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e474fe-7def-4e31-a9ab-8e8811289a70 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.368422 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e474fe-7def-4e31-a9ab-8e8811289a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.398327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-59e474fe-7def-4e31-a9ab-8e8811289a70 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:16.399536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-59e474fe-7def-4e31-a9ab-8e8811289a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:16.409270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1083/2144] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:16.413390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9474a-2769-4cb3-b383-573b99c4b418 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:16.426372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9474a-2769-4cb3-b383-573b99c4b418 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:16.432301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa5aa6-941b-443f-a6ff-6249be0617be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:16.432966 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa5aa6-941b-443f-a6ff-6249be0617be admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.433251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa5aa6-941b-443f-a6ff-6249be0617be admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.433527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa5aa6-941b-443f-a6ff-6249be0617be admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.434296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa5aa6-941b-443f-a6ff-6249be0617be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:19:16.434608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa5aa6-941b-443f-a6ff-6249be0617be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:16.436195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa5aa6-941b-443f-a6ff-6249be0617be admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:16.449460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1062/2145] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:19:16.453554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e474fe-7def-4e31-a9ab-8e8811289a70 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:16.463916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e474fe-7def-4e31-a9ab-8e8811289a70 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:16.468731 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b814e9-89d9-45a9-8ade-e21948f46e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:16.469390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b814e9-89d9-45a9-8ade-e21948f46e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.469676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b814e9-89d9-45a9-8ade-e21948f46e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.470608 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b814e9-89d9-45a9-8ade-e21948f46e80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.471337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b814e9-89d9-45a9-8ade-e21948f46e80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:16.473327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b814e9-89d9-45a9-8ade-e21948f46e80 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:16.510884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1084/2146] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:16 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:16.512420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa5aa6-941b-443f-a6ff-6249be0617be admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:16.527028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa5aa6-941b-443f-a6ff-6249be0617be admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:16.613258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139161-b977-4302-bcd1-994a35c4e022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:16.614255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139161-b977-4302-bcd1-994a35c4e022 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.614402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139161-b977-4302-bcd1-994a35c4e022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.614768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139161-b977-4302-bcd1-994a35c4e022 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.616257 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139161-b977-4302-bcd1-994a35c4e022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:16.618294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139161-b977-4302-bcd1-994a35c4e022 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:16.625281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b814e9-89d9-45a9-8ade-e21948f46e80 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:16.627647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-90b814e9-89d9-45a9-8ade-e21948f46e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:16.644275 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1063/2147] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:16 2022] POST /identity/v3/users => generated 283 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:16.648305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139161-b977-4302-bcd1-994a35c4e022 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:16.654191 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b814e9-89d9-45a9-8ade-e21948f46e80 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:16.659187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c5299-3441-409f-ae4a-863ecebf28a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:16.660536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c5299-3441-409f-ae4a-863ecebf28a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.660838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c5299-3441-409f-ae4a-863ecebf28a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.661199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c5299-3441-409f-ae4a-863ecebf28a9 admin admin] PATH_INFO: `/v3/projects/4b65381065474a2aa42e9ce620816ef3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.662693 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c5299-3441-409f-ae4a-863ecebf28a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b65381065474a2aa42e9ce620816ef3)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:16.665175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c5299-3441-409f-ae4a-863ecebf28a9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:16.666567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139161-b977-4302-bcd1-994a35c4e022 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:16.672777 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a660cc-6a86-41d5-8ed1-a7ff5a3f983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:16.673476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a660cc-6a86-41d5-8ed1-a7ff5a3f983b admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.674038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a660cc-6a86-41d5-8ed1-a7ff5a3f983b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.674373 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a660cc-6a86-41d5-8ed1-a7ff5a3f983b admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.675208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a660cc-6a86-41d5-8ed1-a7ff5a3f983b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:16.677253 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a660cc-6a86-41d5-8ed1-a7ff5a3f983b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:16.691162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1064/2148] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:16.694963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a660cc-6a86-41d5-8ed1-a7ff5a3f983b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:16.707236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a660cc-6a86-41d5-8ed1-a7ff5a3f983b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:16.716865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24372469-3d6f-42f1-be7a-d6111c246a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:16.717535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24372469-3d6f-42f1-be7a-d6111c246a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.717990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24372469-3d6f-42f1-be7a-d6111c246a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.718112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24372469-3d6f-42f1-be7a-d6111c246a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.718985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24372469-3d6f-42f1-be7a-d6111c246a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:16.726754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24372469-3d6f-42f1-be7a-d6111c246a6c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:16.743387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1065/2149] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:16.748393 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d58c5299-3441-409f-ae4a-863ecebf28a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b65381065474a2aa42e9ce620816ef3', 'request_id': 'req-d58c5299-3441-409f-ae4a-863ecebf28a9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.751807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24372469-3d6f-42f1-be7a-d6111c246a6c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:16.755930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d58c5299-3441-409f-ae4a-863ecebf28a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b65381065474a2aa42e9ce620816ef3', 'request_id': 'req-d58c5299-3441-409f-ae4a-863ecebf28a9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.762461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24372469-3d6f-42f1-be7a-d6111c246a6c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:16.763337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d58c5299-3441-409f-ae4a-863ecebf28a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b65381065474a2aa42e9ce620816ef3', 'request_id': 'req-d58c5299-3441-409f-ae4a-863ecebf28a9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.778734 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d58c5299-3441-409f-ae4a-863ecebf28a9 admin admin] The token cache is being invalidate because project 4b65381065474a2aa42e9ce620816ef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:16.779023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d58c5299-3441-409f-ae4a-863ecebf28a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.779704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d58c5299-3441-409f-ae4a-863ecebf28a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:16.785807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1085/2150] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:16 2022] DELETE /identity/v3/projects/4b65381065474a2aa42e9ce620816ef3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:16.787500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfecc6-4e17-42c3-956d-0d60691cf3d3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.787806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfecc6-4e17-42c3-956d-0d60691cf3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.788081 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfecc6-4e17-42c3-956d-0d60691cf3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.817025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-78dfecc6-4e17-42c3-956d-0d60691cf3d3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:16.818271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-78dfecc6-4e17-42c3-956d-0d60691cf3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:16.823165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:16.825481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd08876-a61d-4e56-b163-128c9a94beaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:16.826543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd08876-a61d-4e56-b163-128c9a94beaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.826659 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd08876-a61d-4e56-b163-128c9a94beaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.826917 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd08876-a61d-4e56-b163-128c9a94beaa admin admin] PATH_INFO: `/v3/projects/d731fe9ceba945629944b5434d42b0e7/users/a760104d285045bbbb18aa7d93dd7e0d/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.848736 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd08876-a61d-4e56-b163-128c9a94beaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d731fe9ceba945629944b5434d42b0e7, user_id=a760104d285045bbbb18aa7d93dd7e0d, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:16.852288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd08876-a61d-4e56-b163-128c9a94beaa admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:16.883618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1066/2151] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:16 2022] PUT /identity/v3/projects/d731fe9ceba945629944b5434d42b0e7/users/a760104d285045bbbb18aa7d93dd7e0d/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:16.886398 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f7e2d-a26b-4b0b-9207-20e9838d5122 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.886800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f7e2d-a26b-4b0b-9207-20e9838d5122 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.887450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f7e2d-a26b-4b0b-9207-20e9838d5122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.902588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1086/2152] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:16.904285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fa739-7403-4851-800f-10c4d4670e32 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:16.904577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fa739-7403-4851-800f-10c4d4670e32 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:16.904854 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fa739-7403-4851-800f-10c4d4670e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:16.919451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:16.923056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7a7f7e2d-a26b-4b0b-9207-20e9838d5122 None None] MFA Rules not processed for user `a760104d285045bbbb18aa7d93dd7e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:16.926929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7a7f7e2d-a26b-4b0b-9207-20e9838d5122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:16.938977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c44fa739-7403-4851-800f-10c4d4670e32 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:16.941021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c44fa739-7403-4851-800f-10c4d4670e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:17.002871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1067/2153] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:17.007231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f7e2d-a26b-4b0b-9207-20e9838d5122 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.008504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7a7f7e2d-a26b-4b0b-9207-20e9838d5122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:17.009978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1087/2154] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:17.014469 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fa739-7403-4851-800f-10c4d4670e32 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.024045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f7e2d-a26b-4b0b-9207-20e9838d5122 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.024667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fa739-7403-4851-800f-10c4d4670e32 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.028594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2dd68-2460-4881-a911-9d8ec72fa4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.029224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56464028-9394-47dd-9bed-2316c22bdbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.029369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2dd68-2460-4881-a911-9d8ec72fa4fb admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.029540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2dd68-2460-4881-a911-9d8ec72fa4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.029844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2dd68-2460-4881-a911-9d8ec72fa4fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.029942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56464028-9394-47dd-9bed-2316c22bdbe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.030179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56464028-9394-47dd-9bed-2316c22bdbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.030458 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56464028-9394-47dd-9bed-2316c22bdbe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.030597 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2dd68-2460-4881-a911-9d8ec72fa4fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.031185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56464028-9394-47dd-9bed-2316c22bdbe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:19:17.031493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56464028-9394-47dd-9bed-2316c22bdbe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.032637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2dd68-2460-4881-a911-9d8ec72fa4fb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.033129 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56464028-9394-47dd-9bed-2316c22bdbe8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.045477 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1088/2155] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:19:17.050894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56464028-9394-47dd-9bed-2316c22bdbe8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.059768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1068/2156] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:17 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:17.061268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56464028-9394-47dd-9bed-2316c22bdbe8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.064498 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2dd68-2460-4881-a911-9d8ec72fa4fb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.131569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e6161-af68-40cd-a0bf-97cbf680369e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.131926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2dd68-2460-4881-a911-9d8ec72fa4fb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.132236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e6161-af68-40cd-a0bf-97cbf680369e admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.132583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e6161-af68-40cd-a0bf-97cbf680369e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.132795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e6161-af68-40cd-a0bf-97cbf680369e admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.133485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e6161-af68-40cd-a0bf-97cbf680369e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.135459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e6161-af68-40cd-a0bf-97cbf680369e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.136645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b481ec4-b429-43b3-9000-20fef9fc42a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.137308 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b481ec4-b429-43b3-9000-20fef9fc42a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.137619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b481ec4-b429-43b3-9000-20fef9fc42a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.137930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b481ec4-b429-43b3-9000-20fef9fc42a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.138653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b481ec4-b429-43b3-9000-20fef9fc42a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.140560 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b481ec4-b429-43b3-9000-20fef9fc42a7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.166361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1069/2157] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:17 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:17.169571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1089/2158] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:17 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:17.170458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b481ec4-b429-43b3-9000-20fef9fc42a7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.174415 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e6161-af68-40cd-a0bf-97cbf680369e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.184559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b481ec4-b429-43b3-9000-20fef9fc42a7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.201127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:17.255167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e6161-af68-40cd-a0bf-97cbf680369e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.256674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18d9d0-cdba-4920-b302-1373dd3e6e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.257390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18d9d0-cdba-4920-b302-1373dd3e6e0a admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.257758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18d9d0-cdba-4920-b302-1373dd3e6e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.258116 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18d9d0-cdba-4920-b302-1373dd3e6e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.258226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009274d2-241b-44bd-a5f2-d183dc133a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.258845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009274d2-241b-44bd-a5f2-d183dc133a1a admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.258945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18d9d0-cdba-4920-b302-1373dd3e6e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.259129 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009274d2-241b-44bd-a5f2-d183dc133a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.259403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009274d2-241b-44bd-a5f2-d183dc133a1a admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.260094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009274d2-241b-44bd-a5f2-d183dc133a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.260840 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18d9d0-cdba-4920-b302-1373dd3e6e0a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.262300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009274d2-241b-44bd-a5f2-d183dc133a1a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.280302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1070/2159] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:17.281661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18d9d0-cdba-4920-b302-1373dd3e6e0a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.292687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18d9d0-cdba-4920-b302-1373dd3e6e0a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.301764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1090/2160] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:17 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:17.301941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abec84-fb41-4c80-bbbe-c8ee14f1c8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.301941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abec84-fb41-4c80-bbbe-c8ee14f1c8f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.302217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abec84-fb41-4c80-bbbe-c8ee14f1c8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.302497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abec84-fb41-4c80-bbbe-c8ee14f1c8f3 admin admin] PATH_INFO: `/v3/projects/01c40b26e0144d3097828da56919a9be/users/41163b04d0714cb3ab93f067677fbbba/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.303320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009274d2-241b-44bd-a5f2-d183dc133a1a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.304878 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:17.313975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009274d2-241b-44bd-a5f2-d183dc133a1a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.319152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d84474-555a-438a-a78e-c8b898177058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.319818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abec84-fb41-4c80-bbbe-c8ee14f1c8f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c40b26e0144d3097828da56919a9be, user_id=41163b04d0714cb3ab93f067677fbbba, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.319927 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d84474-555a-438a-a78e-c8b898177058 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.320158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d84474-555a-438a-a78e-c8b898177058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.320450 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d84474-555a-438a-a78e-c8b898177058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.321178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d84474-555a-438a-a78e-c8b898177058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.323067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abec84-fb41-4c80-bbbe-c8ee14f1c8f3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.323148 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d84474-555a-438a-a78e-c8b898177058 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.337641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1091/2161] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:17.337853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1071/2162] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:17 2022] PUT /identity/v3/projects/01c40b26e0144d3097828da56919a9be/users/41163b04d0714cb3ab93f067677fbbba/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:17.339337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d84474-555a-438a-a78e-c8b898177058 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.340908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abec84-fb41-4c80-bbbe-c8ee14f1c8f3 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.352470 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d84474-555a-438a-a78e-c8b898177058 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.407765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abec84-fb41-4c80-bbbe-c8ee14f1c8f3 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.409369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9e7f4-5721-4c3e-9fe1-8377b95f611d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.410083 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9e7f4-5721-4c3e-9fe1-8377b95f611d admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.410368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9e7f4-5721-4c3e-9fe1-8377b95f611d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.410642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9e7f4-5721-4c3e-9fe1-8377b95f611d admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.411366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9e7f4-5721-4c3e-9fe1-8377b95f611d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.412570 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a125c-510b-4eb5-b9cc-d40bb90cbeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.413024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9e7f4-5721-4c3e-9fe1-8377b95f611d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.413232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a125c-510b-4eb5-b9cc-d40bb90cbeac admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.413512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a125c-510b-4eb5-b9cc-d40bb90cbeac admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.413840 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a125c-510b-4eb5-b9cc-d40bb90cbeac admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.414564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a125c-510b-4eb5-b9cc-d40bb90cbeac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.416144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a125c-510b-4eb5-b9cc-d40bb90cbeac admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.430424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1092/2163] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:19:17.430612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1072/2164] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:17.430706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9e7f4-5721-4c3e-9fe1-8377b95f611d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.432025 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a125c-510b-4eb5-b9cc-d40bb90cbeac admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.439829 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9e7f4-5721-4c3e-9fe1-8377b95f611d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.443330 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a125c-510b-4eb5-b9cc-d40bb90cbeac admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.444783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f0bb5-4311-480b-8fcd-f64587c15d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.445710 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f0bb5-4311-480b-8fcd-f64587c15d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.446025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f0bb5-4311-480b-8fcd-f64587c15d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.446301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f0bb5-4311-480b-8fcd-f64587c15d5a admin admin] PATH_INFO: `/v3/projects/0decd2033fa64d0fb04b7d237ea751b4/users/56f61d77ab1f4b2fb85f6ec4ff81faf7/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.447964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c74baf-0ca6-4a95-8d9f-3cc4b0461fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.448843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c74baf-0ca6-4a95-8d9f-3cc4b0461fce admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.449121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c74baf-0ca6-4a95-8d9f-3cc4b0461fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.449393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c74baf-0ca6-4a95-8d9f-3cc4b0461fce admin admin] PATH_INFO: `/v3/projects/01c40b26e0144d3097828da56919a9be/users/41163b04d0714cb3ab93f067677fbbba/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.451412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c74baf-0ca6-4a95-8d9f-3cc4b0461fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c40b26e0144d3097828da56919a9be, user_id=41163b04d0714cb3ab93f067677fbbba, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.454146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c74baf-0ca6-4a95-8d9f-3cc4b0461fce admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.463195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f0bb5-4311-480b-8fcd-f64587c15d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0decd2033fa64d0fb04b7d237ea751b4, user_id=56f61d77ab1f4b2fb85f6ec4ff81faf7, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.465383 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1073/2165] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:17 2022] PUT /identity/v3/projects/01c40b26e0144d3097828da56919a9be/users/41163b04d0714cb3ab93f067677fbbba/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:17.465903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f0bb5-4311-480b-8fcd-f64587c15d5a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.475941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c8da9-637c-414c-aada-36992b8f5f19 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.476232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c8da9-637c-414c-aada-36992b8f5f19 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.476512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c8da9-637c-414c-aada-36992b8f5f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.479554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1093/2166] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:17 2022] PUT /identity/v3/projects/0decd2033fa64d0fb04b7d237ea751b4/users/56f61d77ab1f4b2fb85f6ec4ff81faf7/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:17.491006 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c578d-9c16-49a7-bba3-4d418d97399a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.491288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c578d-9c16-49a7-bba3-4d418d97399a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.491558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c578d-9c16-49a7-bba3-4d418d97399a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.505860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-da7c8da9-637c-414c-aada-36992b8f5f19 None None] MFA Rules not processed for user `41163b04d0714cb3ab93f067677fbbba`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:17.507089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-da7c8da9-637c-414c-aada-36992b8f5f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:17.524552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9d4c578d-9c16-49a7-bba3-4d418d97399a None None] MFA Rules not processed for user `56f61d77ab1f4b2fb85f6ec4ff81faf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:17.525922 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9d4c578d-9c16-49a7-bba3-4d418d97399a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:17.581800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1074/2167] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:17 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:17.592693 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1094/2168] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:17.593093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c8da9-637c-414c-aada-36992b8f5f19 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.594595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-da7c8da9-637c-414c-aada-36992b8f5f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:17.604103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c578d-9c16-49a7-bba3-4d418d97399a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.605479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9d4c578d-9c16-49a7-bba3-4d418d97399a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:17.614693 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:17.656809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c8da9-637c-414c-aada-36992b8f5f19 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.660615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e6e65-096c-4531-a7e3-49849485963d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.661440 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e6e65-096c-4531-a7e3-49849485963d service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.661720 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e6e65-096c-4531-a7e3-49849485963d service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.661988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e6e65-096c-4531-a7e3-49849485963d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.668729 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:17.676310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e6e65-096c-4531-a7e3-49849485963d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.680608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e6e65-096c-4531-a7e3-49849485963d service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.685047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c578d-9c16-49a7-bba3-4d418d97399a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.690848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e89cb1-266b-49b6-b5c3-586005fcb044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.691433 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e89cb1-266b-49b6-b5c3-586005fcb044 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.691716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e89cb1-266b-49b6-b5c3-586005fcb044 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.691990 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e89cb1-266b-49b6-b5c3-586005fcb044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.694759 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1075/2169] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:19:17.702029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e89cb1-266b-49b6-b5c3-586005fcb044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.703644 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e89cb1-266b-49b6-b5c3-586005fcb044 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.714970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1095/2170] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:19:17.782824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e6e65-096c-4531-a7e3-49849485963d service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.801273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e6e65-096c-4531-a7e3-49849485963d service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.805331 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697da240-0795-434a-b74e-bc4d6f1e5864 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.806123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697da240-0795-434a-b74e-bc4d6f1e5864 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.806487 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697da240-0795-434a-b74e-bc4d6f1e5864 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.806766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697da240-0795-434a-b74e-bc4d6f1e5864 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.875437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697da240-0795-434a-b74e-bc4d6f1e5864 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.877209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697da240-0795-434a-b74e-bc4d6f1e5864 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.889897 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1076/2171] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:21.482257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e89cb1-266b-49b6-b5c3-586005fcb044 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:21.576468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e89cb1-266b-49b6-b5c3-586005fcb044 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:21.594333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:21.595300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:21.595584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:21.595859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] PATH_INFO: `/v3/users/56f61d77ab1f4b2fb85f6ec4ff81faf7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:21.597154 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56f61d77ab1f4b2fb85f6ec4ff81faf7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:21.599199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:21.662353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56f61d77ab1f4b2fb85f6ec4ff81faf7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:21.672182 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56f61d77ab1f4b2fb85f6ec4ff81faf7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:21.673345 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56f61d77ab1f4b2fb85f6ec4ff81faf7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:21.674155 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56f61d77ab1f4b2fb85f6ec4ff81faf7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:21.690688 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56f61d77ab1f4b2fb85f6ec4ff81faf7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:21.704826 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1096/2172] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:21 2022] DELETE /identity/v3/users/56f61d77ab1f4b2fb85f6ec4ff81faf7 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:21.709197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cefeaa-a98f-47d7-b0bb-9df649266790 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:21.709609 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cefeaa-a98f-47d7-b0bb-9df649266790 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:21.710005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cefeaa-a98f-47d7-b0bb-9df649266790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:21.732928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-22cefeaa-a98f-47d7-b0bb-9df649266790 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:21.734194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-22cefeaa-a98f-47d7-b0bb-9df649266790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:21.818861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1077/2173] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:21.828899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:21.831071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:21.869782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:21.911434 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9f49a-46b9-4821-974d-36470b54f2d5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:21.915074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c535fb-1084-495e-8342-dec87ac15a64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:21.915738 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c535fb-1084-495e-8342-dec87ac15a64 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:21.916017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c535fb-1084-495e-8342-dec87ac15a64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:21.916287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c535fb-1084-495e-8342-dec87ac15a64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:21.926725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c535fb-1084-495e-8342-dec87ac15a64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:21.928194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c535fb-1084-495e-8342-dec87ac15a64 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:21.939491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1097/2174] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:22.401359 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cefeaa-a98f-47d7-b0bb-9df649266790 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:22.402895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-22cefeaa-a98f-47d7-b0bb-9df649266790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:22.419951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cefeaa-a98f-47d7-b0bb-9df649266790 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:22.425238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959c5a4-519b-4240-af10-22838b5dfc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:22.426339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959c5a4-519b-4240-af10-22838b5dfc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:22.427018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959c5a4-519b-4240-af10-22838b5dfc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:22.427018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959c5a4-519b-4240-af10-22838b5dfc8f admin admin] PATH_INFO: `/v3/projects/0decd2033fa64d0fb04b7d237ea751b4` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:22.428717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959c5a4-519b-4240-af10-22838b5dfc8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0decd2033fa64d0fb04b7d237ea751b4)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:22.430710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959c5a4-519b-4240-af10-22838b5dfc8f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:22.490699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7959c5a4-519b-4240-af10-22838b5dfc8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0decd2033fa64d0fb04b7d237ea751b4', 'request_id': 'req-7959c5a4-519b-4240-af10-22838b5dfc8f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:22.501257 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7959c5a4-519b-4240-af10-22838b5dfc8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0decd2033fa64d0fb04b7d237ea751b4', 'request_id': 'req-7959c5a4-519b-4240-af10-22838b5dfc8f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:22.507874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7959c5a4-519b-4240-af10-22838b5dfc8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0decd2033fa64d0fb04b7d237ea751b4', 'request_id': 'req-7959c5a4-519b-4240-af10-22838b5dfc8f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:22.519315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7959c5a4-519b-4240-af10-22838b5dfc8f admin admin] The token cache is being invalidate because project 0decd2033fa64d0fb04b7d237ea751b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:22.519605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7959c5a4-519b-4240-af10-22838b5dfc8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:22.520274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7959c5a4-519b-4240-af10-22838b5dfc8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:22.522702 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1078/2175] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:22 2022] DELETE /identity/v3/projects/0decd2033fa64d0fb04b7d237ea751b4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:22.527572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6ffd3-7c3e-4cb9-abb8-108e9d56a12f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:22.529488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6ffd3-7c3e-4cb9-abb8-108e9d56a12f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:22.529941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6ffd3-7c3e-4cb9-abb8-108e9d56a12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:22.555431 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-00f6ffd3-7c3e-4cb9-abb8-108e9d56a12f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:22.558174 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-00f6ffd3-7c3e-4cb9-abb8-108e9d56a12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:22.642686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1098/2176] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:22.651140 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc535f-57e3-4e7f-ba4e-ec1b6e54c7e6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:22.651500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc535f-57e3-4e7f-ba4e-ec1b6e54c7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:22.651784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc535f-57e3-4e7f-ba4e-ec1b6e54c7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:22.669993 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b1bc535f-57e3-4e7f-ba4e-ec1b6e54c7e6 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:22.671199 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b1bc535f-57e3-4e7f-ba4e-ec1b6e54c7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:22.679787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1079/2177] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:22.683992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6ffd3-7c3e-4cb9-abb8-108e9d56a12f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:22.694587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6ffd3-7c3e-4cb9-abb8-108e9d56a12f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:22.699244 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067dacb-4c8b-4983-86bd-c7edba55dc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:22.699906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067dacb-4c8b-4983-86bd-c7edba55dc8f admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:22.700187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067dacb-4c8b-4983-86bd-c7edba55dc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:22.700496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067dacb-4c8b-4983-86bd-c7edba55dc8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:22.701223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067dacb-4c8b-4983-86bd-c7edba55dc8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:19:22.701532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067dacb-4c8b-4983-86bd-c7edba55dc8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:22.703234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067dacb-4c8b-4983-86bd-c7edba55dc8f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:22.719274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1099/2178] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:19:22.720607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc535f-57e3-4e7f-ba4e-ec1b6e54c7e6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:22.730065 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc535f-57e3-4e7f-ba4e-ec1b6e54c7e6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:22.737544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea3f32-812b-4b33-ac46-390bf8dc789f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:22.738236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea3f32-812b-4b33-ac46-390bf8dc789f admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:22.738513 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea3f32-812b-4b33-ac46-390bf8dc789f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:22.738779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea3f32-812b-4b33-ac46-390bf8dc789f admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:22.739530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea3f32-812b-4b33-ac46-390bf8dc789f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:22.741429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea3f32-812b-4b33-ac46-390bf8dc789f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:22.745956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:22.781610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1080/2179] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:22 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:22.785502 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067dacb-4c8b-4983-86bd-c7edba55dc8f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:22.795861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067dacb-4c8b-4983-86bd-c7edba55dc8f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:22.865004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff879ec-d067-4780-9e23-b4d21a0e1afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:22.865677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff879ec-d067-4780-9e23-b4d21a0e1afa admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:22.865989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff879ec-d067-4780-9e23-b4d21a0e1afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:22.866262 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff879ec-d067-4780-9e23-b4d21a0e1afa admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:22.866957 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff879ec-d067-4780-9e23-b4d21a0e1afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:22.868926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff879ec-d067-4780-9e23-b4d21a0e1afa admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:22.893428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1100/2180] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:22 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:22.897271 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea3f32-812b-4b33-ac46-390bf8dc789f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:22.907519 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea3f32-812b-4b33-ac46-390bf8dc789f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:22.913588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e8f8a-0ab5-4c2f-a04e-f648be216609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:22.914288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e8f8a-0ab5-4c2f-a04e-f648be216609 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:22.914701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e8f8a-0ab5-4c2f-a04e-f648be216609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:22.914992 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e8f8a-0ab5-4c2f-a04e-f648be216609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:22.915724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e8f8a-0ab5-4c2f-a04e-f648be216609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:22.917393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e8f8a-0ab5-4c2f-a04e-f648be216609 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:22.928543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1081/2181] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:22.932548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff879ec-d067-4780-9e23-b4d21a0e1afa admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:22.943477 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff879ec-d067-4780-9e23-b4d21a0e1afa admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:22.948404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42627a8d-8270-4a4f-a417-b0bfec5bc2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:22.949067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42627a8d-8270-4a4f-a417-b0bfec5bc2dc admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:22.949349 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42627a8d-8270-4a4f-a417-b0bfec5bc2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:22.949621 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42627a8d-8270-4a4f-a417-b0bfec5bc2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:22.950380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42627a8d-8270-4a4f-a417-b0bfec5bc2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:22.951985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42627a8d-8270-4a4f-a417-b0bfec5bc2dc admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:22.962968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1101/2182] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:22.966854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e8f8a-0ab5-4c2f-a04e-f648be216609 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:22.977273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e8f8a-0ab5-4c2f-a04e-f648be216609 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:22.981751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ddbbe-5c05-4026-911b-c836eaea1d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:22.982629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ddbbe-5c05-4026-911b-c836eaea1d7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:22.982906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ddbbe-5c05-4026-911b-c836eaea1d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:22.983174 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ddbbe-5c05-4026-911b-c836eaea1d7e admin admin] PATH_INFO: `/v3/projects/24b643e4d06847748e4090a93c0f0057/users/a1373aff4fc64b6481a175aff012276c/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:22.998196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ddbbe-5c05-4026-911b-c836eaea1d7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b643e4d06847748e4090a93c0f0057, user_id=a1373aff4fc64b6481a175aff012276c, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:23.001411 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ddbbe-5c05-4026-911b-c836eaea1d7e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:23.018339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1082/2183] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:22 2022] PUT /identity/v3/projects/24b643e4d06847748e4090a93c0f0057/users/a1373aff4fc64b6481a175aff012276c/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:23.028615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941447b1-db22-4bdf-9576-3c222f8943b4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:23.028907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941447b1-db22-4bdf-9576-3c222f8943b4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:23.029187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941447b1-db22-4bdf-9576-3c222f8943b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:23.061700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-941447b1-db22-4bdf-9576-3c222f8943b4 None None] MFA Rules not processed for user `a1373aff4fc64b6481a175aff012276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:23.062965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-941447b1-db22-4bdf-9576-3c222f8943b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:23.085337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:23.134363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1102/2184] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:23.146359 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ddbbe-5c05-4026-911b-c836eaea1d7e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:23.148072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ac8ddbbe-5c05-4026-911b-c836eaea1d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:23.209479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ddbbe-5c05-4026-911b-c836eaea1d7e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:23.213448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beac403-d55b-497d-97f2-047f361530a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a67120485f74f8bb0ff188cd3abd21c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:23.216091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beac403-d55b-497d-97f2-047f361530a9 service glance] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:23.216091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beac403-d55b-497d-97f2-047f361530a9 service glance] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:23.216091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beac403-d55b-497d-97f2-047f361530a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:23.224578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beac403-d55b-497d-97f2-047f361530a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:23.226036 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beac403-d55b-497d-97f2-047f361530a9 service glance] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:23.236507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1083/2185] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:31.524885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941447b1-db22-4bdf-9576-3c222f8943b4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:31.527031 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-941447b1-db22-4bdf-9576-3c222f8943b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:31.724020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941447b1-db22-4bdf-9576-3c222f8943b4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:31.728639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:31.729504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:31.729813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:31.730114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] PATH_INFO: `/v3/users/a760104d285045bbbb18aa7d93dd7e0d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:31.731288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a760104d285045bbbb18aa7d93dd7e0d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:31.733211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:31.791256 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a760104d285045bbbb18aa7d93dd7e0d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:31.799914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a760104d285045bbbb18aa7d93dd7e0d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:31.800868 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a760104d285045bbbb18aa7d93dd7e0d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:31.804693 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a760104d285045bbbb18aa7d93dd7e0d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:31.815171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:31.823781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a760104d285045bbbb18aa7d93dd7e0d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:31.844292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1103/2186] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:31 2022] DELETE /identity/v3/users/a760104d285045bbbb18aa7d93dd7e0d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:31.849138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6611ee-486e-428a-9212-b0ffe6e00a24 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:31.849657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6611ee-486e-428a-9212-b0ffe6e00a24 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:31.849779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6611ee-486e-428a-9212-b0ffe6e00a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:31.884385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-df6611ee-486e-428a-9212-b0ffe6e00a24 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:31.885924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-df6611ee-486e-428a-9212-b0ffe6e00a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:31.982801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1084/2187] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:31.995298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:31.997513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:32.116230 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6c9ea-88ab-4e70-a835-8ad44b01e382 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:32.120828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b13b90-c68f-4f8d-89f3-b928dadad685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:32.121481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b13b90-c68f-4f8d-89f3-b928dadad685 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:32.121785 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b13b90-c68f-4f8d-89f3-b928dadad685 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:32.122058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b13b90-c68f-4f8d-89f3-b928dadad685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:32.136512 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b13b90-c68f-4f8d-89f3-b928dadad685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:32.137928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b13b90-c68f-4f8d-89f3-b928dadad685 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:32.149487 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1104/2188] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:19:32.450137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6611ee-486e-428a-9212-b0ffe6e00a24 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:32.451916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-df6611ee-486e-428a-9212-b0ffe6e00a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:32.468325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6611ee-486e-428a-9212-b0ffe6e00a24 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:32.476050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:32.476930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:32.477209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:32.477488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] PATH_INFO: `/v3/projects/d731fe9ceba945629944b5434d42b0e7` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:32.479180 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d731fe9ceba945629944b5434d42b0e7)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:32.482654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:32.550389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:32.555560 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd731fe9ceba945629944b5434d42b0e7', 'request_id': 'req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:32.568152 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd731fe9ceba945629944b5434d42b0e7', 'request_id': 'req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:32.576872 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd731fe9ceba945629944b5434d42b0e7', 'request_id': 'req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:32.592427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] The token cache is being invalidate because project d731fe9ceba945629944b5434d42b0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:32.592716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:32.593570 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:32.597049 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1085/2189] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:32 2022] DELETE /identity/v3/projects/d731fe9ceba945629944b5434d42b0e7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:32.600372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b13b90-c68f-4f8d-89f3-b928dadad685 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:32.616844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b13b90-c68f-4f8d-89f3-b928dadad685 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:32.675820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873a798-cc4b-43c1-bd6a-a629a3a87524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:32.676754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873a798-cc4b-43c1-bd6a-a629a3a87524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:32.677036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873a798-cc4b-43c1-bd6a-a629a3a87524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:32.677305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873a798-cc4b-43c1-bd6a-a629a3a87524 admin admin] PATH_INFO: `/v3/users/41163b04d0714cb3ab93f067677fbbba` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:32.678584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873a798-cc4b-43c1-bd6a-a629a3a87524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41163b04d0714cb3ab93f067677fbbba)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:32.680544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873a798-cc4b-43c1-bd6a-a629a3a87524 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:32.734757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e873a798-cc4b-43c1-bd6a-a629a3a87524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41163b04d0714cb3ab93f067677fbbba'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:32.739065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:32.745784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e873a798-cc4b-43c1-bd6a-a629a3a87524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41163b04d0714cb3ab93f067677fbbba'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:32.746433 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e873a798-cc4b-43c1-bd6a-a629a3a87524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41163b04d0714cb3ab93f067677fbbba'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:32.747078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e873a798-cc4b-43c1-bd6a-a629a3a87524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41163b04d0714cb3ab93f067677fbbba'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:32.763998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e873a798-cc4b-43c1-bd6a-a629a3a87524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41163b04d0714cb3ab93f067677fbbba'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:32.779833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1105/2190] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:32 2022] DELETE /identity/v3/users/41163b04d0714cb3ab93f067677fbbba => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:33.116723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:33.118184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:33.137543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff37ebe-bc5c-4053-8b1b-d93e1d5b3f23 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:33.205489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8118483-e79e-4973-ad16-158a07c41b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:33.206378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8118483-e79e-4973-ad16-158a07c41b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:33.206659 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8118483-e79e-4973-ad16-158a07c41b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:33.206929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8118483-e79e-4973-ad16-158a07c41b7a admin admin] PATH_INFO: `/v3/projects/01c40b26e0144d3097828da56919a9be` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:33.208116 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8118483-e79e-4973-ad16-158a07c41b7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=01c40b26e0144d3097828da56919a9be)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:33.210088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8118483-e79e-4973-ad16-158a07c41b7a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:33.264892 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b8118483-e79e-4973-ad16-158a07c41b7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01c40b26e0144d3097828da56919a9be', 'request_id': 'req-b8118483-e79e-4973-ad16-158a07c41b7a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:33.273698 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b8118483-e79e-4973-ad16-158a07c41b7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01c40b26e0144d3097828da56919a9be', 'request_id': 'req-b8118483-e79e-4973-ad16-158a07c41b7a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:33.280137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b8118483-e79e-4973-ad16-158a07c41b7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01c40b26e0144d3097828da56919a9be', 'request_id': 'req-b8118483-e79e-4973-ad16-158a07c41b7a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:33.291048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b8118483-e79e-4973-ad16-158a07c41b7a admin admin] The token cache is being invalidate because project 01c40b26e0144d3097828da56919a9be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:33.291332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b8118483-e79e-4973-ad16-158a07c41b7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:33.291966 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b8118483-e79e-4973-ad16-158a07c41b7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:33.293587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1086/2191] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:33 2022] DELETE /identity/v3/projects/01c40b26e0144d3097828da56919a9be => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:33.299898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507769b-903c-4ea4-8707-90e1fc4653d9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:33.300190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507769b-903c-4ea4-8707-90e1fc4653d9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:33.300463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507769b-903c-4ea4-8707-90e1fc4653d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:33.337452 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b507769b-903c-4ea4-8707-90e1fc4653d9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:33.338688 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b507769b-903c-4ea4-8707-90e1fc4653d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:33.399390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1106/2192] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:33.408021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d699558-49d1-4fff-be56-24d9abf05498 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:33.408366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d699558-49d1-4fff-be56-24d9abf05498 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:33.408538 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d699558-49d1-4fff-be56-24d9abf05498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:33.429670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1d699558-49d1-4fff-be56-24d9abf05498 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:33.430907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1d699558-49d1-4fff-be56-24d9abf05498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:33.439591 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1087/2193] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:33.444061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507769b-903c-4ea4-8707-90e1fc4653d9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:33.453964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507769b-903c-4ea4-8707-90e1fc4653d9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:33.458823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e76e24-db1b-4b50-842d-99d40abf142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:33.459561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e76e24-db1b-4b50-842d-99d40abf142a admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:33.459860 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e76e24-db1b-4b50-842d-99d40abf142a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:33.460203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e76e24-db1b-4b50-842d-99d40abf142a admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:33.460999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e76e24-db1b-4b50-842d-99d40abf142a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:19:33.461306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e76e24-db1b-4b50-842d-99d40abf142a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:33.463126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e76e24-db1b-4b50-842d-99d40abf142a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:33.478755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1107/2194] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:19:33.483118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d699558-49d1-4fff-be56-24d9abf05498 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:33.494193 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d699558-49d1-4fff-be56-24d9abf05498 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:33.499947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b993d83-1354-4cf5-a5ce-e793aa52df90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:33.500697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b993d83-1354-4cf5-a5ce-e793aa52df90 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:33.500999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b993d83-1354-4cf5-a5ce-e793aa52df90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:33.501507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b993d83-1354-4cf5-a5ce-e793aa52df90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:33.502397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b993d83-1354-4cf5-a5ce-e793aa52df90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:33.504383 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b993d83-1354-4cf5-a5ce-e793aa52df90 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:33.532988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1088/2195] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:33 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:33.535170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e76e24-db1b-4b50-842d-99d40abf142a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:33.545115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e76e24-db1b-4b50-842d-99d40abf142a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:33.610448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc1e5c-affb-462e-a38f-33f015f2b54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:33.611334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc1e5c-affb-462e-a38f-33f015f2b54c admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:33.611429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc1e5c-affb-462e-a38f-33f015f2b54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:33.611838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc1e5c-affb-462e-a38f-33f015f2b54c admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:33.612592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc1e5c-affb-462e-a38f-33f015f2b54c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:33.614564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc1e5c-affb-462e-a38f-33f015f2b54c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:33.636683 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1108/2196] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:33 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:33.640868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b993d83-1354-4cf5-a5ce-e793aa52df90 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:33.652200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b993d83-1354-4cf5-a5ce-e793aa52df90 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:33.657200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0105e6-9520-432b-8b4a-f9e5458066f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:33.658594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0105e6-9520-432b-8b4a-f9e5458066f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:33.658982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0105e6-9520-432b-8b4a-f9e5458066f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:33.659262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0105e6-9520-432b-8b4a-f9e5458066f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:33.659984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0105e6-9520-432b-8b4a-f9e5458066f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:33.661572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0105e6-9520-432b-8b4a-f9e5458066f7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:33.673169 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1089/2197] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:33.678405 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc1e5c-affb-462e-a38f-33f015f2b54c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:33.688345 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc1e5c-affb-462e-a38f-33f015f2b54c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:33.693444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456545fd-0950-4fd1-8511-a12ab2e06e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:33.694127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456545fd-0950-4fd1-8511-a12ab2e06e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:33.694406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456545fd-0950-4fd1-8511-a12ab2e06e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:33.694676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456545fd-0950-4fd1-8511-a12ab2e06e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:33.695387 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456545fd-0950-4fd1-8511-a12ab2e06e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:33.696983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456545fd-0950-4fd1-8511-a12ab2e06e1d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:33.711769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1109/2198] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:33.713745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0105e6-9520-432b-8b4a-f9e5458066f7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:33.725323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0105e6-9520-432b-8b4a-f9e5458066f7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:33.730009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99eb38-098d-48fc-b9dc-3dc9202b82cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:33.730883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99eb38-098d-48fc-b9dc-3dc9202b82cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:33.731163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99eb38-098d-48fc-b9dc-3dc9202b82cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:33.731454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99eb38-098d-48fc-b9dc-3dc9202b82cd admin admin] PATH_INFO: `/v3/projects/f88666a7b4724d8192ec41a9a4cea28f/users/3a66e950128f4b61b72d5331308686d3/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:33.747123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99eb38-098d-48fc-b9dc-3dc9202b82cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f88666a7b4724d8192ec41a9a4cea28f, user_id=3a66e950128f4b61b72d5331308686d3, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:33.750118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99eb38-098d-48fc-b9dc-3dc9202b82cd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:33.758219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:33.774865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1090/2199] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:33 2022] PUT /identity/v3/projects/f88666a7b4724d8192ec41a9a4cea28f/users/3a66e950128f4b61b72d5331308686d3/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:33.776599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b65223-914c-4a05-a876-c7a8d430902c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:33.776887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b65223-914c-4a05-a876-c7a8d430902c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:33.777590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b65223-914c-4a05-a876-c7a8d430902c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:33.812816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d9b65223-914c-4a05-a876-c7a8d430902c None None] MFA Rules not processed for user `3a66e950128f4b61b72d5331308686d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:33.815321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d9b65223-914c-4a05-a876-c7a8d430902c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:33.825186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:33.896497 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1110/2200] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:33.901284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99eb38-098d-48fc-b9dc-3dc9202b82cd admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:34.021239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99eb38-098d-48fc-b9dc-3dc9202b82cd admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:34.027280 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca1627-3054-46bf-a63a-fef02a4f4b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:34.028072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca1627-3054-46bf-a63a-fef02a4f4b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:34.028790 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca1627-3054-46bf-a63a-fef02a4f4b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:34.029132 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca1627-3054-46bf-a63a-fef02a4f4b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:34.030005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca1627-3054-46bf-a63a-fef02a4f4b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:34.032636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca1627-3054-46bf-a63a-fef02a4f4b58 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:34.072713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1091/2201] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:33 2022] POST /identity/v3/projects => generated 393 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:34.074351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b65223-914c-4a05-a876-c7a8d430902c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:34.092771 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b65223-914c-4a05-a876-c7a8d430902c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:34.155729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fb35d-b5c1-4ae8-87f0-dfaa9c5c4a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:34.156466 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fb35d-b5c1-4ae8-87f0-dfaa9c5c4a2d admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:34.156748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fb35d-b5c1-4ae8-87f0-dfaa9c5c4a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:34.157020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fb35d-b5c1-4ae8-87f0-dfaa9c5c4a2d admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:34.158231 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fb35d-b5c1-4ae8-87f0-dfaa9c5c4a2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:34.160807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fb35d-b5c1-4ae8-87f0-dfaa9c5c4a2d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:34.183625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1111/2202] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:34 2022] POST /identity/v3/users => generated 303 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:34.187446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca1627-3054-46bf-a63a-fef02a4f4b58 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:34.200574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca1627-3054-46bf-a63a-fef02a4f4b58 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:34.210370 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e8972-8a49-4f1e-94dd-1dc222a9ada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:34.211351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e8972-8a49-4f1e-94dd-1dc222a9ada7 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:34.211706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e8972-8a49-4f1e-94dd-1dc222a9ada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:34.212067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e8972-8a49-4f1e-94dd-1dc222a9ada7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:34.213092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e8972-8a49-4f1e-94dd-1dc222a9ada7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:34.215237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e8972-8a49-4f1e-94dd-1dc222a9ada7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:34.228481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1092/2203] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:34.233084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fb35d-b5c1-4ae8-87f0-dfaa9c5c4a2d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:34.252720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fb35d-b5c1-4ae8-87f0-dfaa9c5c4a2d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:34.264130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927997ea-7e9e-4134-a35d-972f88487d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:34.265037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927997ea-7e9e-4134-a35d-972f88487d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:34.265356 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927997ea-7e9e-4134-a35d-972f88487d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:34.265581 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927997ea-7e9e-4134-a35d-972f88487d6c admin admin] PATH_INFO: `/v3/projects/e19ef6f69d184572847e53e8f09872fd/users/c2dfde20d126480caa29f4e06f47df61/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:34.294604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927997ea-7e9e-4134-a35d-972f88487d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19ef6f69d184572847e53e8f09872fd, user_id=c2dfde20d126480caa29f4e06f47df61, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:34.299110 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927997ea-7e9e-4134-a35d-972f88487d6c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:34.321002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1112/2204] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:34 2022] PUT /identity/v3/projects/e19ef6f69d184572847e53e8f09872fd/users/c2dfde20d126480caa29f4e06f47df61/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:34.324719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e8972-8a49-4f1e-94dd-1dc222a9ada7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:34.335395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e8972-8a49-4f1e-94dd-1dc222a9ada7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:34.399569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f820f7f-3d93-41bb-bb02-35c9318aa3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:34.400257 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f820f7f-3d93-41bb-bb02-35c9318aa3eb admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:34.400548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f820f7f-3d93-41bb-bb02-35c9318aa3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:34.400822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f820f7f-3d93-41bb-bb02-35c9318aa3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:34.401548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f820f7f-3d93-41bb-bb02-35c9318aa3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:34.403183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f820f7f-3d93-41bb-bb02-35c9318aa3eb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:34.414955 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1093/2205] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:34.418892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927997ea-7e9e-4134-a35d-972f88487d6c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:34.430494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927997ea-7e9e-4134-a35d-972f88487d6c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:34.435383 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7f503-cdbf-411c-a1e3-06eac9d8f27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:34.436301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7f503-cdbf-411c-a1e3-06eac9d8f27e admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:34.436576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7f503-cdbf-411c-a1e3-06eac9d8f27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:34.436845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7f503-cdbf-411c-a1e3-06eac9d8f27e admin admin] PATH_INFO: `/v3/projects/e19ef6f69d184572847e53e8f09872fd/users/c2dfde20d126480caa29f4e06f47df61/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:34.439601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7f503-cdbf-411c-a1e3-06eac9d8f27e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19ef6f69d184572847e53e8f09872fd, user_id=c2dfde20d126480caa29f4e06f47df61, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:34.442608 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7f503-cdbf-411c-a1e3-06eac9d8f27e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:34.453718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1113/2206] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:34 2022] PUT /identity/v3/projects/e19ef6f69d184572847e53e8f09872fd/users/c2dfde20d126480caa29f4e06f47df61/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:34.465653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c8cac-4a5e-452c-a19a-210859c0c7b0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:34.465958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c8cac-4a5e-452c-a19a-210859c0c7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:34.466350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c8cac-4a5e-452c-a19a-210859c0c7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:34.499473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b27c8cac-4a5e-452c-a19a-210859c0c7b0 None None] MFA Rules not processed for user `c2dfde20d126480caa29f4e06f47df61`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:34.500722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b27c8cac-4a5e-452c-a19a-210859c0c7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:34.575925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:34.588204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1094/2207] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:34.600090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7f503-cdbf-411c-a1e3-06eac9d8f27e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:34.601824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8dc7f503-cdbf-411c-a1e3-06eac9d8f27e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:34.660402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7f503-cdbf-411c-a1e3-06eac9d8f27e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:34.664092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199a966-0491-4837-82c8-5f06fa133b23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:34.664733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199a966-0491-4837-82c8-5f06fa133b23 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:34.665017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199a966-0491-4837-82c8-5f06fa133b23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:34.665294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199a966-0491-4837-82c8-5f06fa133b23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:34.674533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199a966-0491-4837-82c8-5f06fa133b23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:34.675972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199a966-0491-4837-82c8-5f06fa133b23 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:34.685558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1114/2208] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:34.731414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c8cac-4a5e-452c-a19a-210859c0c7b0 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:34.732793 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b27c8cac-4a5e-452c-a19a-210859c0c7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:34.745463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c8cac-4a5e-452c-a19a-210859c0c7b0 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:34.749080 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f5240-b1ec-4177-9e2d-c6dd8c52cc2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:34.749712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f5240-b1ec-4177-9e2d-c6dd8c52cc2d service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:34.750021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f5240-b1ec-4177-9e2d-c6dd8c52cc2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:34.750292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f5240-b1ec-4177-9e2d-c6dd8c52cc2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:34.809210 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f5240-b1ec-4177-9e2d-c6dd8c52cc2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:34.811100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f5240-b1ec-4177-9e2d-c6dd8c52cc2d service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:34.824124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1095/2209] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:35.301779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199a966-0491-4837-82c8-5f06fa133b23 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:35.303299 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6199a966-0491-4837-82c8-5f06fa133b23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:35.308753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:35.380890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199a966-0491-4837-82c8-5f06fa133b23 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:35.385350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:35.386251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:35.386530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:35.386796 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] PATH_INFO: `/v3/users/3dfa4ee464054e3e87c3f4784b8931a2` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:35.387966 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dfa4ee464054e3e87c3f4784b8931a2)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:35.389914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:35.438014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dfa4ee464054e3e87c3f4784b8931a2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:35.446344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dfa4ee464054e3e87c3f4784b8931a2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:35.447034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dfa4ee464054e3e87c3f4784b8931a2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:35.447756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dfa4ee464054e3e87c3f4784b8931a2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:35.460724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dfa4ee464054e3e87c3f4784b8931a2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:35.474191 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1115/2210] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:35 2022] DELETE /identity/v3/users/3dfa4ee464054e3e87c3f4784b8931a2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:35.479503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f77d79-20ad-42ba-adc5-e62132bd9266 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:35.480005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f77d79-20ad-42ba-adc5-e62132bd9266 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:35.480005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f77d79-20ad-42ba-adc5-e62132bd9266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:35.498026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-00f77d79-20ad-42ba-adc5-e62132bd9266 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:35.499521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-00f77d79-20ad-42ba-adc5-e62132bd9266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:35.560959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1096/2211] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:35.571114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:35.572587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:35.635146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561983e8-520e-4eb5-ab53-5f43d82f3ab5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:35.638794 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5ffd5-376a-45ad-9023-0672d99972c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:35.639435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5ffd5-376a-45ad-9023-0672d99972c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:35.639745 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5ffd5-376a-45ad-9023-0672d99972c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:35.640018 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5ffd5-376a-45ad-9023-0672d99972c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:35.648943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5ffd5-376a-45ad-9023-0672d99972c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:35.650437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5ffd5-376a-45ad-9023-0672d99972c5 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:35.660412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1116/2212] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:35.932308 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f77d79-20ad-42ba-adc5-e62132bd9266 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:35.933739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-00f77d79-20ad-42ba-adc5-e62132bd9266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:35.948072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f77d79-20ad-42ba-adc5-e62132bd9266 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:35.952722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9d8d3-d5e6-4700-9539-c644422e3ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:35.955198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9d8d3-d5e6-4700-9539-c644422e3ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:35.955583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9d8d3-d5e6-4700-9539-c644422e3ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:35.955859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9d8d3-d5e6-4700-9539-c644422e3ca5 admin admin] PATH_INFO: `/v3/projects/2df16509d14441cabe3fb9c7806a2639` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:35.957081 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9d8d3-d5e6-4700-9539-c644422e3ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2df16509d14441cabe3fb9c7806a2639)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:35.959064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9d8d3-d5e6-4700-9539-c644422e3ca5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:36.008367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-76d9d8d3-d5e6-4700-9539-c644422e3ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2df16509d14441cabe3fb9c7806a2639', 'request_id': 'req-76d9d8d3-d5e6-4700-9539-c644422e3ca5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:36.016412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-76d9d8d3-d5e6-4700-9539-c644422e3ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2df16509d14441cabe3fb9c7806a2639', 'request_id': 'req-76d9d8d3-d5e6-4700-9539-c644422e3ca5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:36.023418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-76d9d8d3-d5e6-4700-9539-c644422e3ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2df16509d14441cabe3fb9c7806a2639', 'request_id': 'req-76d9d8d3-d5e6-4700-9539-c644422e3ca5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:36.035224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-76d9d8d3-d5e6-4700-9539-c644422e3ca5 admin admin] The token cache is being invalidate because project 2df16509d14441cabe3fb9c7806a2639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:36.035553 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-76d9d8d3-d5e6-4700-9539-c644422e3ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:36.036254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-76d9d8d3-d5e6-4700-9539-c644422e3ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:36.037975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1097/2213] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:35 2022] DELETE /identity/v3/projects/2df16509d14441cabe3fb9c7806a2639 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:36.043196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f1810-aeeb-484c-ad71-b4d86e59ed7f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.043688 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f1810-aeeb-484c-ad71-b4d86e59ed7f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.043820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f1810-aeeb-484c-ad71-b4d86e59ed7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.061972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-272f1810-aeeb-484c-ad71-b4d86e59ed7f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:36.063294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-272f1810-aeeb-484c-ad71-b4d86e59ed7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:36.083508 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:36.127015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1117/2214] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:36.134822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d978a4b-1409-4e07-96fb-511d52c0581b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.135125 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d978a4b-1409-4e07-96fb-511d52c0581b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.135408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d978a4b-1409-4e07-96fb-511d52c0581b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.162762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5d978a4b-1409-4e07-96fb-511d52c0581b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:36.164122 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5d978a4b-1409-4e07-96fb-511d52c0581b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:36.172745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1098/2215] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:36.177453 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f1810-aeeb-484c-ad71-b4d86e59ed7f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:36.187146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f1810-aeeb-484c-ad71-b4d86e59ed7f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:36.192198 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b725e-8320-487d-acd0-9fe0c7ffda33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:36.192861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b725e-8320-487d-acd0-9fe0c7ffda33 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.193133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b725e-8320-487d-acd0-9fe0c7ffda33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.193410 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b725e-8320-487d-acd0-9fe0c7ffda33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.194171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b725e-8320-487d-acd0-9fe0c7ffda33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:19:36.194477 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b725e-8320-487d-acd0-9fe0c7ffda33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:36.196297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b725e-8320-487d-acd0-9fe0c7ffda33 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:36.209900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1118/2216] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:19:36.214424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d978a4b-1409-4e07-96fb-511d52c0581b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:36.224007 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d978a4b-1409-4e07-96fb-511d52c0581b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:36.229114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe2504-79dc-4522-9cad-6460fb947ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:36.229819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe2504-79dc-4522-9cad-6460fb947ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.230233 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe2504-79dc-4522-9cad-6460fb947ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.230512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe2504-79dc-4522-9cad-6460fb947ba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.231232 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe2504-79dc-4522-9cad-6460fb947ba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:36.233121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe2504-79dc-4522-9cad-6460fb947ba0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:36.237673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:36.274058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b725e-8320-487d-acd0-9fe0c7ffda33 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:36.279615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-606b725e-8320-487d-acd0-9fe0c7ffda33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:36.289069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1099/2217] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:36 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:36.289989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe2504-79dc-4522-9cad-6460fb947ba0 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:36.293962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b725e-8320-487d-acd0-9fe0c7ffda33 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:36.385657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe2504-79dc-4522-9cad-6460fb947ba0 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:36.393641 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ee0c3-1c7b-466d-8c5e-e5ba1dd857ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:36.394365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ee0c3-1c7b-466d-8c5e-e5ba1dd857ef admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.394644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ee0c3-1c7b-466d-8c5e-e5ba1dd857ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.394916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ee0c3-1c7b-466d-8c5e-e5ba1dd857ef admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.395612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ee0c3-1c7b-466d-8c5e-e5ba1dd857ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:36.397530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ee0c3-1c7b-466d-8c5e-e5ba1dd857ef admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:36.428458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1100/2218] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:36 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:36.434130 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ee0c3-1c7b-466d-8c5e-e5ba1dd857ef admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:36.440980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cbf63-ccbe-4273-b482-6af13ecdc28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:36.443429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cbf63-ccbe-4273-b482-6af13ecdc28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.444176 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cbf63-ccbe-4273-b482-6af13ecdc28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.444580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cbf63-ccbe-4273-b482-6af13ecdc28b admin admin] PATH_INFO: `/v3/users/959b31fe78fc4e7583f49714802a927a` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.446542 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cbf63-ccbe-4273-b482-6af13ecdc28b admin admin] RBAC: Authorizing `identity:delete_user(user_id=959b31fe78fc4e7583f49714802a927a)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:36.449291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cbf63-ccbe-4273-b482-6af13ecdc28b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:36.459442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ee0c3-1c7b-466d-8c5e-e5ba1dd857ef admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:36.475180 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0e5af-3901-4a0f-b591-b48f165b5294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:36.476221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0e5af-3901-4a0f-b591-b48f165b5294 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.476595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0e5af-3901-4a0f-b591-b48f165b5294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.476930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0e5af-3901-4a0f-b591-b48f165b5294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.477752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0e5af-3901-4a0f-b591-b48f165b5294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:36.479477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0e5af-3901-4a0f-b591-b48f165b5294 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:36.491386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1101/2219] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:36.495332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0e5af-3901-4a0f-b591-b48f165b5294 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:36.506338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0e5af-3901-4a0f-b591-b48f165b5294 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:36.511962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c8d21-52b9-4d7c-8d3b-47c3b7db951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:36.512627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c8d21-52b9-4d7c-8d3b-47c3b7db951a admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.512906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c8d21-52b9-4d7c-8d3b-47c3b7db951a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.513183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c8d21-52b9-4d7c-8d3b-47c3b7db951a admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.513931 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c8d21-52b9-4d7c-8d3b-47c3b7db951a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:36.515533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c8d21-52b9-4d7c-8d3b-47c3b7db951a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:36.529603 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1102/2220] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:19:36.534404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c8d21-52b9-4d7c-8d3b-47c3b7db951a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:36.535704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ff0cbf63-ccbe-4273-b482-6af13ecdc28b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '959b31fe78fc4e7583f49714802a927a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:36.546644 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ff0cbf63-ccbe-4273-b482-6af13ecdc28b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '959b31fe78fc4e7583f49714802a927a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:36.548236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ff0cbf63-ccbe-4273-b482-6af13ecdc28b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '959b31fe78fc4e7583f49714802a927a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:36.549355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ff0cbf63-ccbe-4273-b482-6af13ecdc28b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '959b31fe78fc4e7583f49714802a927a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:36.550078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c8d21-52b9-4d7c-8d3b-47c3b7db951a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:36.554836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49725445-601a-45a3-ac1f-cf9b3cf9aa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:36.555842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49725445-601a-45a3-ac1f-cf9b3cf9aa4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.556263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49725445-601a-45a3-ac1f-cf9b3cf9aa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.556436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49725445-601a-45a3-ac1f-cf9b3cf9aa4a admin admin] PATH_INFO: `/v3/projects/d08521617c884fe9a67609887f0ddaae/users/5462b6a8d72749b9a07cd52ff8cb29d1/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.563303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ff0cbf63-ccbe-4273-b482-6af13ecdc28b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '959b31fe78fc4e7583f49714802a927a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:36.577386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49725445-601a-45a3-ac1f-cf9b3cf9aa4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d08521617c884fe9a67609887f0ddaae, user_id=5462b6a8d72749b9a07cd52ff8cb29d1, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:36.581207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49725445-601a-45a3-ac1f-cf9b3cf9aa4a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:36.586330 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1119/2221] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:36 2022] DELETE /identity/v3/users/959b31fe78fc4e7583f49714802a927a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:36.588377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebf090-7948-4c6a-95dc-e30cca1f5fdd None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.588705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebf090-7948-4c6a-95dc-e30cca1f5fdd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.588984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebf090-7948-4c6a-95dc-e30cca1f5fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.597217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1103/2222] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:36 2022] PUT /identity/v3/projects/d08521617c884fe9a67609887f0ddaae/users/5462b6a8d72749b9a07cd52ff8cb29d1/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:36.602442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbb066-c3aa-4f04-b7e4-820a44ff33d3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.602595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbb066-c3aa-4f04-b7e4-820a44ff33d3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.602834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbb066-c3aa-4f04-b7e4-820a44ff33d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.609370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b0ebf090-7948-4c6a-95dc-e30cca1f5fdd None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:36.611320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b0ebf090-7948-4c6a-95dc-e30cca1f5fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:36.624615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-51dbb066-c3aa-4f04-b7e4-820a44ff33d3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:36.626167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-51dbb066-c3aa-4f04-b7e4-820a44ff33d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:36.639122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:36.684189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1120/2223] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:36.694144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebf090-7948-4c6a-95dc-e30cca1f5fdd None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:36.696021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b0ebf090-7948-4c6a-95dc-e30cca1f5fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:36.731943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1104/2224] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:36.743724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbb066-c3aa-4f04-b7e4-820a44ff33d3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:36.746072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-51dbb066-c3aa-4f04-b7e4-820a44ff33d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:36.776753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebf090-7948-4c6a-95dc-e30cca1f5fdd None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:36.780454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a533d5d-fc51-4aa6-8c5c-801eded3874d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:36.781102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a533d5d-fc51-4aa6-8c5c-801eded3874d service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.781380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a533d5d-fc51-4aa6-8c5c-801eded3874d service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.781647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a533d5d-fc51-4aa6-8c5c-801eded3874d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.792024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a533d5d-fc51-4aa6-8c5c-801eded3874d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:36.793635 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a533d5d-fc51-4aa6-8c5c-801eded3874d service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:36.794751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:36.809941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1121/2225] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:36.832355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbb066-c3aa-4f04-b7e4-820a44ff33d3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:36.838096 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7217a0-4bd8-4d86-ae1f-eb2f6b5d2042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:36.839185 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7217a0-4bd8-4d86-ae1f-eb2f6b5d2042 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:36.839535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7217a0-4bd8-4d86-ae1f-eb2f6b5d2042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:36.840020 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7217a0-4bd8-4d86-ae1f-eb2f6b5d2042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:36.852962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7217a0-4bd8-4d86-ae1f-eb2f6b5d2042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:36.854565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7217a0-4bd8-4d86-ae1f-eb2f6b5d2042 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:36.871809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1105/2226] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:37.168833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a533d5d-fc51-4aa6-8c5c-801eded3874d service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:37.172360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5a533d5d-fc51-4aa6-8c5c-801eded3874d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:37.203112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a533d5d-fc51-4aa6-8c5c-801eded3874d service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:37.218092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2bb9d-0f78-4c70-8093-c41ccad755e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:37.219256 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2bb9d-0f78-4c70-8093-c41ccad755e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:37.219532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2bb9d-0f78-4c70-8093-c41ccad755e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:37.219968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2bb9d-0f78-4c70-8093-c41ccad755e6 admin admin] PATH_INFO: `/v3/projects/098290c4499240e5bb671d21c8f42a8a` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:37.221410 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2bb9d-0f78-4c70-8093-c41ccad755e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=098290c4499240e5bb671d21c8f42a8a)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:37.229587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2bb9d-0f78-4c70-8093-c41ccad755e6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:37.327882 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-22c2bb9d-0f78-4c70-8093-c41ccad755e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '098290c4499240e5bb671d21c8f42a8a', 'request_id': 'req-22c2bb9d-0f78-4c70-8093-c41ccad755e6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:37.341427 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-22c2bb9d-0f78-4c70-8093-c41ccad755e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '098290c4499240e5bb671d21c8f42a8a', 'request_id': 'req-22c2bb9d-0f78-4c70-8093-c41ccad755e6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:37.364099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-22c2bb9d-0f78-4c70-8093-c41ccad755e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '098290c4499240e5bb671d21c8f42a8a', 'request_id': 'req-22c2bb9d-0f78-4c70-8093-c41ccad755e6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:37.381873 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-22c2bb9d-0f78-4c70-8093-c41ccad755e6 admin admin] The token cache is being invalidate because project 098290c4499240e5bb671d21c8f42a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:37.382168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-22c2bb9d-0f78-4c70-8093-c41ccad755e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:37.382898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-22c2bb9d-0f78-4c70-8093-c41ccad755e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:37.400634 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1122/2227] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:37 2022] DELETE /identity/v3/projects/098290c4499240e5bb671d21c8f42a8a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:38.474941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7217a0-4bd8-4d86-ae1f-eb2f6b5d2042 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:38.476943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ac7217a0-4bd8-4d86-ae1f-eb2f6b5d2042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:38.490842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7217a0-4bd8-4d86-ae1f-eb2f6b5d2042 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:38.578523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:38.579776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:38.580264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:38.580683 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] PATH_INFO: `/v3/users/3a66e950128f4b61b72d5331308686d3` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:38.587096 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a66e950128f4b61b72d5331308686d3)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:38.589386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:38.663564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a66e950128f4b61b72d5331308686d3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:38.678071 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a66e950128f4b61b72d5331308686d3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:38.692463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a66e950128f4b61b72d5331308686d3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:38.693205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a66e950128f4b61b72d5331308686d3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:38.701526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a66e950128f4b61b72d5331308686d3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:38.710320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1106/2228] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:38 2022] DELETE /identity/v3/users/3a66e950128f4b61b72d5331308686d3 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:38.716513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180e05e-8f7d-4665-a742-883fcd8b8590 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:38.717649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180e05e-8f7d-4665-a742-883fcd8b8590 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:38.718136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180e05e-8f7d-4665-a742-883fcd8b8590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:38.754136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a180e05e-8f7d-4665-a742-883fcd8b8590 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:38.760992 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a180e05e-8f7d-4665-a742-883fcd8b8590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:38.871218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:38.882921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1123/2229] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:38.892392 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:38.894594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:38.976421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36982d8-97fc-4671-b7e9-05974444e87f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:38.980098 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8a786-4725-493f-ba52-f3ad743f75d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:38.980750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8a786-4725-493f-ba52-f3ad743f75d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:38.981032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8a786-4725-493f-ba52-f3ad743f75d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:38.981306 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8a786-4725-493f-ba52-f3ad743f75d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:38.989409 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:38.992008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8a786-4725-493f-ba52-f3ad743f75d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:38.993815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8a786-4725-493f-ba52-f3ad743f75d5 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:39.005615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1107/2230] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:39.323409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180e05e-8f7d-4665-a742-883fcd8b8590 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:39.323409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a180e05e-8f7d-4665-a742-883fcd8b8590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:39.338946 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180e05e-8f7d-4665-a742-883fcd8b8590 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:39.345408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:39.346356 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:39.346798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:39.347083 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] PATH_INFO: `/v3/projects/f88666a7b4724d8192ec41a9a4cea28f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:39.348754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f88666a7b4724d8192ec41a9a4cea28f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:39.350947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:39.415176 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f88666a7b4724d8192ec41a9a4cea28f', 'request_id': 'req-74592910-a71e-4253-84c8-9eef16b346b8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:39.430361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f88666a7b4724d8192ec41a9a4cea28f', 'request_id': 'req-74592910-a71e-4253-84c8-9eef16b346b8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:39.436530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f88666a7b4724d8192ec41a9a4cea28f', 'request_id': 'req-74592910-a71e-4253-84c8-9eef16b346b8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:39.454844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] The token cache is being invalidate because project f88666a7b4724d8192ec41a9a4cea28f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:39.455372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:39.456216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:39.458042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1124/2231] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:39 2022] DELETE /identity/v3/projects/f88666a7b4724d8192ec41a9a4cea28f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:39.462045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8a786-4725-493f-ba52-f3ad743f75d5 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:39.472936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8a786-4725-493f-ba52-f3ad743f75d5 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:39.549480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dff786-fbbf-4152-ba3b-30f4965172f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:39.550636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dff786-fbbf-4152-ba3b-30f4965172f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:39.551034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dff786-fbbf-4152-ba3b-30f4965172f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:39.551420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dff786-fbbf-4152-ba3b-30f4965172f5 admin admin] PATH_INFO: `/v3/users/c2dfde20d126480caa29f4e06f47df61` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:39.553143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dff786-fbbf-4152-ba3b-30f4965172f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2dfde20d126480caa29f4e06f47df61)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:39.555785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dff786-fbbf-4152-ba3b-30f4965172f5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:39.627002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-95dff786-fbbf-4152-ba3b-30f4965172f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2dfde20d126480caa29f4e06f47df61'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:39.639123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-95dff786-fbbf-4152-ba3b-30f4965172f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2dfde20d126480caa29f4e06f47df61'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:39.655729 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-95dff786-fbbf-4152-ba3b-30f4965172f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2dfde20d126480caa29f4e06f47df61'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:39.656466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-95dff786-fbbf-4152-ba3b-30f4965172f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2dfde20d126480caa29f4e06f47df61'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:39.664803 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-95dff786-fbbf-4152-ba3b-30f4965172f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2dfde20d126480caa29f4e06f47df61'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:39.677493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1108/2232] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:39 2022] DELETE /identity/v3/users/c2dfde20d126480caa29f4e06f47df61 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:40.367576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:40.369191 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:40.383142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74592910-a71e-4253-84c8-9eef16b346b8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:40.443323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42989a95-afac-412f-aa60-f6c4cae53432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:40.444837 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42989a95-afac-412f-aa60-f6c4cae53432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:40.445321 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42989a95-afac-412f-aa60-f6c4cae53432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:40.445614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42989a95-afac-412f-aa60-f6c4cae53432 admin admin] PATH_INFO: `/v3/projects/e19ef6f69d184572847e53e8f09872fd` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:40.446878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42989a95-afac-412f-aa60-f6c4cae53432 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19ef6f69d184572847e53e8f09872fd)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:40.448828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42989a95-afac-412f-aa60-f6c4cae53432 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:40.511685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-42989a95-afac-412f-aa60-f6c4cae53432 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19ef6f69d184572847e53e8f09872fd', 'request_id': 'req-42989a95-afac-412f-aa60-f6c4cae53432'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:40.519201 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-42989a95-afac-412f-aa60-f6c4cae53432 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19ef6f69d184572847e53e8f09872fd', 'request_id': 'req-42989a95-afac-412f-aa60-f6c4cae53432'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:40.525122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-42989a95-afac-412f-aa60-f6c4cae53432 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19ef6f69d184572847e53e8f09872fd', 'request_id': 'req-42989a95-afac-412f-aa60-f6c4cae53432'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:40.537793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-42989a95-afac-412f-aa60-f6c4cae53432 admin admin] The token cache is being invalidate because project e19ef6f69d184572847e53e8f09872fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:40.538406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-42989a95-afac-412f-aa60-f6c4cae53432 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:40.539038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-42989a95-afac-412f-aa60-f6c4cae53432 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:40.540626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1125/2233] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:40 2022] DELETE /identity/v3/projects/e19ef6f69d184572847e53e8f09872fd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:40.545526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae353c-cbdf-4342-a2c7-1a96da246bd9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:40.545979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae353c-cbdf-4342-a2c7-1a96da246bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:40.546328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae353c-cbdf-4342-a2c7-1a96da246bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:40.567852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-50ae353c-cbdf-4342-a2c7-1a96da246bd9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:40.569316 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-50ae353c-cbdf-4342-a2c7-1a96da246bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:40.639393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1109/2234] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:40.647319 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546eca7-fa2e-4073-a9f6-dc7cfb04dfe2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:40.647612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546eca7-fa2e-4073-a9f6-dc7cfb04dfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:40.647891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546eca7-fa2e-4073-a9f6-dc7cfb04dfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:40.666774 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1546eca7-fa2e-4073-a9f6-dc7cfb04dfe2 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:40.667994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1546eca7-fa2e-4073-a9f6-dc7cfb04dfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:40.676942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1126/2235] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:40.681114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae353c-cbdf-4342-a2c7-1a96da246bd9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:40.690989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae353c-cbdf-4342-a2c7-1a96da246bd9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:40.695576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc05b5-878c-4446-bc51-2a17cbae25e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:40.696227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc05b5-878c-4446-bc51-2a17cbae25e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:40.696511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc05b5-878c-4446-bc51-2a17cbae25e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:40.696778 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc05b5-878c-4446-bc51-2a17cbae25e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:40.697515 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc05b5-878c-4446-bc51-2a17cbae25e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:19:40.697854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc05b5-878c-4446-bc51-2a17cbae25e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:40.699447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc05b5-878c-4446-bc51-2a17cbae25e7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:40.712066 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1110/2236] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:19:40.716476 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546eca7-fa2e-4073-a9f6-dc7cfb04dfe2 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:40.723209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:40.727734 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546eca7-fa2e-4073-a9f6-dc7cfb04dfe2 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:40.733622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b37f43-a60a-4bff-aee0-e682a1eaafb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:40.734995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b37f43-a60a-4bff-aee0-e682a1eaafb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:40.736355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b37f43-a60a-4bff-aee0-e682a1eaafb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:40.736938 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b37f43-a60a-4bff-aee0-e682a1eaafb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:40.737793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b37f43-a60a-4bff-aee0-e682a1eaafb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:40.740424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b37f43-a60a-4bff-aee0-e682a1eaafb8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:40.779090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1127/2237] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:40 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:40.780871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc05b5-878c-4446-bc51-2a17cbae25e7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:40.790768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:40.807994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff52bfc-c1d3-4f41-8ae4-46c7136b7235 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:40.808339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff52bfc-c1d3-4f41-8ae4-46c7136b7235 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:40.808662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff52bfc-c1d3-4f41-8ae4-46c7136b7235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:40.811599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc05b5-878c-4446-bc51-2a17cbae25e7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:40.835848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-cff52bfc-c1d3-4f41-8ae4-46c7136b7235 None None] MFA Rules not processed for user `8da0a5a09a6041d68c5e445af1a492f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:40.837372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cff52bfc-c1d3-4f41-8ae4-46c7136b7235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:40.895291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df289e48-f729-4963-8477-7efbc9b872d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:40.895582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df289e48-f729-4963-8477-7efbc9b872d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:40.895865 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df289e48-f729-4963-8477-7efbc9b872d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:40.896136 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df289e48-f729-4963-8477-7efbc9b872d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:40.896970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df289e48-f729-4963-8477-7efbc9b872d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:40.899104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df289e48-f729-4963-8477-7efbc9b872d2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:40.917858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1128/2238] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:40.933760 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1111/2239] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:40 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:40.934940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff52bfc-c1d3-4f41-8ae4-46c7136b7235 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:40.955549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff52bfc-c1d3-4f41-8ae4-46c7136b7235 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:40.960512 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8dad1-fb9e-4995-82dd-07de3b6e26b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:40.961501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8dad1-fb9e-4995-82dd-07de3b6e26b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:40.961883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8dad1-fb9e-4995-82dd-07de3b6e26b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:40.962228 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8dad1-fb9e-4995-82dd-07de3b6e26b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:40.963026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8dad1-fb9e-4995-82dd-07de3b6e26b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:40.964705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8dad1-fb9e-4995-82dd-07de3b6e26b2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:40.978099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1129/2240] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:40.984052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df289e48-f729-4963-8477-7efbc9b872d2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:40.995547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df289e48-f729-4963-8477-7efbc9b872d2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:41.000546 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7d7ee-9f22-4b9c-8ed3-ef9cc28ebe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:41.001674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7d7ee-9f22-4b9c-8ed3-ef9cc28ebe95 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:41.002177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7d7ee-9f22-4b9c-8ed3-ef9cc28ebe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:41.002616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7d7ee-9f22-4b9c-8ed3-ef9cc28ebe95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:41.003466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7d7ee-9f22-4b9c-8ed3-ef9cc28ebe95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:41.005175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7d7ee-9f22-4b9c-8ed3-ef9cc28ebe95 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:41.021227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1112/2241] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:41.025251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8dad1-fb9e-4995-82dd-07de3b6e26b2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:41.036755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8dad1-fb9e-4995-82dd-07de3b6e26b2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:41.041818 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a0b7a-42cb-4f6e-a02b-ee8103831aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:41.042872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a0b7a-42cb-4f6e-a02b-ee8103831aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:41.043955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a0b7a-42cb-4f6e-a02b-ee8103831aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:41.044464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a0b7a-42cb-4f6e-a02b-ee8103831aa4 admin admin] PATH_INFO: `/v3/projects/42f3f6fc30e544239e23b4ea6191fe36/users/617b5182a15945d590058a691d141fe6/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:41.067715 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a0b7a-42cb-4f6e-a02b-ee8103831aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f3f6fc30e544239e23b4ea6191fe36, user_id=617b5182a15945d590058a691d141fe6, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:41.071290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a0b7a-42cb-4f6e-a02b-ee8103831aa4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:41.104501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1130/2242] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:41 2022] PUT /identity/v3/projects/42f3f6fc30e544239e23b4ea6191fe36/users/617b5182a15945d590058a691d141fe6/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:41.118602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37599c9f-77e9-468b-a5de-23c30f1bee15 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:41.118941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37599c9f-77e9-468b-a5de-23c30f1bee15 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:41.120401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37599c9f-77e9-468b-a5de-23c30f1bee15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:41.159973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-37599c9f-77e9-468b-a5de-23c30f1bee15 None None] MFA Rules not processed for user `617b5182a15945d590058a691d141fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:41.161972 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-37599c9f-77e9-468b-a5de-23c30f1bee15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:41.258436 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1113/2243] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:41.260861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a0b7a-42cb-4f6e-a02b-ee8103831aa4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:41.332140 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a0b7a-42cb-4f6e-a02b-ee8103831aa4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:41.337148 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2192572-d981-446d-8722-b6dd3160a357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:41.337948 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2192572-d981-446d-8722-b6dd3160a357 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:41.338305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2192572-d981-446d-8722-b6dd3160a357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:41.338971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2192572-d981-446d-8722-b6dd3160a357 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:41.339415 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2192572-d981-446d-8722-b6dd3160a357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:41.341788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2192572-d981-446d-8722-b6dd3160a357 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:41.372966 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1131/2244] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:41 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:41.380012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37599c9f-77e9-468b-a5de-23c30f1bee15 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:41.398470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37599c9f-77e9-468b-a5de-23c30f1bee15 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:41.496294 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d77048-ee32-4a2d-a22d-c285c2aa2340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:41.497262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d77048-ee32-4a2d-a22d-c285c2aa2340 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:41.497710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d77048-ee32-4a2d-a22d-c285c2aa2340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:41.498268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d77048-ee32-4a2d-a22d-c285c2aa2340 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:41.499523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d77048-ee32-4a2d-a22d-c285c2aa2340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:41.503867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d77048-ee32-4a2d-a22d-c285c2aa2340 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:41.532848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:41.543200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1114/2245] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:41 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:41.546287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2192572-d981-446d-8722-b6dd3160a357 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:41.557777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2192572-d981-446d-8722-b6dd3160a357 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:41.562937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830f7bf-6f1f-4553-a2fc-f01c391bc1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:41.564413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830f7bf-6f1f-4553-a2fc-f01c391bc1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:41.564695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830f7bf-6f1f-4553-a2fc-f01c391bc1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:41.565059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830f7bf-6f1f-4553-a2fc-f01c391bc1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:41.565831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830f7bf-6f1f-4553-a2fc-f01c391bc1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:41.567769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830f7bf-6f1f-4553-a2fc-f01c391bc1d0 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:41.581786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1132/2246] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:41.583432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d77048-ee32-4a2d-a22d-c285c2aa2340 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:41.594301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d77048-ee32-4a2d-a22d-c285c2aa2340 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:41.601605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aad38-2007-4c70-9f19-983f4e6f328a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:41.602311 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aad38-2007-4c70-9f19-983f4e6f328a admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:41.602598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aad38-2007-4c70-9f19-983f4e6f328a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:41.602963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aad38-2007-4c70-9f19-983f4e6f328a admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:41.603757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9aad38-2007-4c70-9f19-983f4e6f328a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:41.605533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9aad38-2007-4c70-9f19-983f4e6f328a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:41.620594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1115/2247] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:41.624615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830f7bf-6f1f-4553-a2fc-f01c391bc1d0 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:41.637989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830f7bf-6f1f-4553-a2fc-f01c391bc1d0 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:41.646837 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99c2b7-974f-46ae-9d1c-d3391db6f8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:41.647788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99c2b7-974f-46ae-9d1c-d3391db6f8b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:41.648293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99c2b7-974f-46ae-9d1c-d3391db6f8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:41.648576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99c2b7-974f-46ae-9d1c-d3391db6f8b4 admin admin] PATH_INFO: `/v3/projects/57acd2f223b24dff8ece49b71d1b873b/users/0e63efdebc144e08923c32689e0b5feb/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:41.674352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99c2b7-974f-46ae-9d1c-d3391db6f8b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57acd2f223b24dff8ece49b71d1b873b, user_id=0e63efdebc144e08923c32689e0b5feb, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:41.677989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99c2b7-974f-46ae-9d1c-d3391db6f8b4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:41.705218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1133/2248] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:41 2022] PUT /identity/v3/projects/57acd2f223b24dff8ece49b71d1b873b/users/0e63efdebc144e08923c32689e0b5feb/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:41.717329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9186b-1c49-4e7d-8068-e47c8fab7410 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:41.717620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9186b-1c49-4e7d-8068-e47c8fab7410 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:41.717927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9186b-1c49-4e7d-8068-e47c8fab7410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:41.748774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e2d9186b-1c49-4e7d-8068-e47c8fab7410 None None] MFA Rules not processed for user `0e63efdebc144e08923c32689e0b5feb`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:41.750071 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e2d9186b-1c49-4e7d-8068-e47c8fab7410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:41.872202 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1116/2249] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:41.881750 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99c2b7-974f-46ae-9d1c-d3391db6f8b4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:41.881750 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:41.960502 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99c2b7-974f-46ae-9d1c-d3391db6f8b4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:41.978569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c92169-b9ef-43a0-9a90-df34b08d7e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:41.980452 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c92169-b9ef-43a0-9a90-df34b08d7e04 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:41.980452 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c92169-b9ef-43a0-9a90-df34b08d7e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:41.980803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c92169-b9ef-43a0-9a90-df34b08d7e04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:41.982294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c92169-b9ef-43a0-9a90-df34b08d7e04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:41.985033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c92169-b9ef-43a0-9a90-df34b08d7e04 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:42.025330 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1134/2250] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:41 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:42.026776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9186b-1c49-4e7d-8068-e47c8fab7410 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:42.041691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9186b-1c49-4e7d-8068-e47c8fab7410 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:42.143538 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67678892-5d4b-47ab-83db-9e0e607a9b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:42.144309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67678892-5d4b-47ab-83db-9e0e607a9b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:42.144661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67678892-5d4b-47ab-83db-9e0e607a9b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:42.145002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67678892-5d4b-47ab-83db-9e0e607a9b06 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:42.145792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67678892-5d4b-47ab-83db-9e0e607a9b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:42.147787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67678892-5d4b-47ab-83db-9e0e607a9b06 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:42.176996 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1117/2251] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:42 2022] POST /identity/v3/users => generated 295 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:42.179463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c92169-b9ef-43a0-9a90-df34b08d7e04 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:42.191696 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c92169-b9ef-43a0-9a90-df34b08d7e04 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:42.198151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0a1dc-872a-47b7-9e67-985a1ba21b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:42.198828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0a1dc-872a-47b7-9e67-985a1ba21b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:42.199095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0a1dc-872a-47b7-9e67-985a1ba21b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:42.199509 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0a1dc-872a-47b7-9e67-985a1ba21b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:42.200292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0a1dc-872a-47b7-9e67-985a1ba21b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:42.202120 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0a1dc-872a-47b7-9e67-985a1ba21b68 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:42.214712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1135/2252] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:42.218980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67678892-5d4b-47ab-83db-9e0e607a9b06 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:42.232505 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67678892-5d4b-47ab-83db-9e0e607a9b06 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:42.239499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa57e08-7b4d-45e3-b009-2e0cb754a7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:42.241236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa57e08-7b4d-45e3-b009-2e0cb754a7af admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:42.241681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa57e08-7b4d-45e3-b009-2e0cb754a7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:42.242011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa57e08-7b4d-45e3-b009-2e0cb754a7af admin admin] PATH_INFO: `/v3/projects/31e0b8b8ad704632afe48895c962fea9/users/3caeeee0b7ca4f8193483f3f5bbf53ac/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:42.263484 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa57e08-7b4d-45e3-b009-2e0cb754a7af admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e0b8b8ad704632afe48895c962fea9, user_id=3caeeee0b7ca4f8193483f3f5bbf53ac, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:42.268659 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa57e08-7b4d-45e3-b009-2e0cb754a7af admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:42.301136 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1118/2253] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:42 2022] PUT /identity/v3/projects/31e0b8b8ad704632afe48895c962fea9/users/3caeeee0b7ca4f8193483f3f5bbf53ac/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:42.304821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0a1dc-872a-47b7-9e67-985a1ba21b68 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:42.317939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0a1dc-872a-47b7-9e67-985a1ba21b68 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:42.390374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259e057-aaf3-4c09-9913-48d7c4a6b83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:42.391126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259e057-aaf3-4c09-9913-48d7c4a6b83f admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:42.391532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259e057-aaf3-4c09-9913-48d7c4a6b83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:42.391875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259e057-aaf3-4c09-9913-48d7c4a6b83f admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:42.392733 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259e057-aaf3-4c09-9913-48d7c4a6b83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:42.394580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259e057-aaf3-4c09-9913-48d7c4a6b83f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:42.412050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1136/2254] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:42.417706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa57e08-7b4d-45e3-b009-2e0cb754a7af admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:42.432756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa57e08-7b4d-45e3-b009-2e0cb754a7af admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:42.439117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5d556-dd6a-4238-8d27-be4de138ef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:42.440152 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5d556-dd6a-4238-8d27-be4de138ef49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:42.440539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5d556-dd6a-4238-8d27-be4de138ef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:42.440913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5d556-dd6a-4238-8d27-be4de138ef49 admin admin] PATH_INFO: `/v3/projects/31e0b8b8ad704632afe48895c962fea9/users/3caeeee0b7ca4f8193483f3f5bbf53ac/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:42.443267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5d556-dd6a-4238-8d27-be4de138ef49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e0b8b8ad704632afe48895c962fea9, user_id=3caeeee0b7ca4f8193483f3f5bbf53ac, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:42.446559 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5d556-dd6a-4238-8d27-be4de138ef49 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:42.459964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1119/2255] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:42 2022] PUT /identity/v3/projects/31e0b8b8ad704632afe48895c962fea9/users/3caeeee0b7ca4f8193483f3f5bbf53ac/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:42.471303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48481b20-3c3b-4cba-a1c0-e5268c5c7e5b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:42.471594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48481b20-3c3b-4cba-a1c0-e5268c5c7e5b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:42.471872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48481b20-3c3b-4cba-a1c0-e5268c5c7e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:42.502434 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-48481b20-3c3b-4cba-a1c0-e5268c5c7e5b None None] MFA Rules not processed for user `3caeeee0b7ca4f8193483f3f5bbf53ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:42.503674 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-48481b20-3c3b-4cba-a1c0-e5268c5c7e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:42.652855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:42.675442 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1137/2256] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:42.692226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5d556-dd6a-4238-8d27-be4de138ef49 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:42.695297 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-34d5d556-dd6a-4238-8d27-be4de138ef49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:42.765333 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:42.802242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5d556-dd6a-4238-8d27-be4de138ef49 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:42.806746 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dede301-4920-490a-b1ea-bcb8c22a5d1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:42.807808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dede301-4920-490a-b1ea-bcb8c22a5d1b service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:42.808937 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dede301-4920-490a-b1ea-bcb8c22a5d1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:42.809479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dede301-4920-490a-b1ea-bcb8c22a5d1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:42.821921 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dede301-4920-490a-b1ea-bcb8c22a5d1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:42.824483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dede301-4920-490a-b1ea-bcb8c22a5d1b service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:42.838856 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1120/2257] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:42.909910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48481b20-3c3b-4cba-a1c0-e5268c5c7e5b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:42.925389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48481b20-3c3b-4cba-a1c0-e5268c5c7e5b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:42.934065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac289d-d46e-423f-889e-b7aed56f19d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:42.934739 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac289d-d46e-423f-889e-b7aed56f19d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:42.935024 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac289d-d46e-423f-889e-b7aed56f19d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:42.935300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac289d-d46e-423f-889e-b7aed56f19d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:43.072696 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac289d-d46e-423f-889e-b7aed56f19d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:43.074871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac289d-d46e-423f-889e-b7aed56f19d8 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:43.094381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1138/2258] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:44.098738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b5824-b23b-4eda-b633-c4696b3c5143 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:44.099803 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b5824-b23b-4eda-b633-c4696b3c5143 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:44.100393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b5824-b23b-4eda-b633-c4696b3c5143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:44.150646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0d9b5824-b23b-4eda-b633-c4696b3c5143 None None] MFA Rules not processed for user `5462b6a8d72749b9a07cd52ff8cb29d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:44.152530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0d9b5824-b23b-4eda-b633-c4696b3c5143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:44.241039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1121/2259] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:44.245079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac289d-d46e-423f-889e-b7aed56f19d8 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:44.247071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-efac289d-d46e-423f-889e-b7aed56f19d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:44.349561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac289d-d46e-423f-889e-b7aed56f19d8 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:44.358344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78947aea-38db-4ac9-b359-26805016d84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:44.359249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78947aea-38db-4ac9-b359-26805016d84b admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:44.359718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78947aea-38db-4ac9-b359-26805016d84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:44.360197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78947aea-38db-4ac9-b359-26805016d84b admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:44.361637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78947aea-38db-4ac9-b359-26805016d84b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:44.364685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78947aea-38db-4ac9-b359-26805016d84b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:44.403103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1139/2260] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:44 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:44.407241 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b5824-b23b-4eda-b633-c4696b3c5143 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:44.420624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b5824-b23b-4eda-b633-c4696b3c5143 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:44.491777 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85293bcd-0ab2-46e0-b49b-bc1b1874694f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:44.492632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85293bcd-0ab2-46e0-b49b-bc1b1874694f admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:44.493010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85293bcd-0ab2-46e0-b49b-bc1b1874694f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:44.493646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85293bcd-0ab2-46e0-b49b-bc1b1874694f admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:44.494211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85293bcd-0ab2-46e0-b49b-bc1b1874694f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:44.496239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85293bcd-0ab2-46e0-b49b-bc1b1874694f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:44.519323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1122/2261] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:44 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:44.523431 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78947aea-38db-4ac9-b359-26805016d84b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:44.536128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78947aea-38db-4ac9-b359-26805016d84b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:44.541756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23776fdf-6d19-46a0-ac4d-b2739ef38a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:44.542504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23776fdf-6d19-46a0-ac4d-b2739ef38a56 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:44.542851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23776fdf-6d19-46a0-ac4d-b2739ef38a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:44.543334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23776fdf-6d19-46a0-ac4d-b2739ef38a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:44.544298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23776fdf-6d19-46a0-ac4d-b2739ef38a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:44.546077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23776fdf-6d19-46a0-ac4d-b2739ef38a56 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:44.572560 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1140/2262] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:44.577287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85293bcd-0ab2-46e0-b49b-bc1b1874694f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:44.592722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85293bcd-0ab2-46e0-b49b-bc1b1874694f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:44.618468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9e891-e2ab-4267-a528-e7ae3755fc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:44.619377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9e891-e2ab-4267-a528-e7ae3755fc12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:44.619669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9e891-e2ab-4267-a528-e7ae3755fc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:44.620008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9e891-e2ab-4267-a528-e7ae3755fc12 admin admin] PATH_INFO: `/v3/projects/6ea0f3f9ce1949beb362a242a84d9400/users/747fb356fcab40e2b96926f3a120f022/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:44.645190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9e891-e2ab-4267-a528-e7ae3755fc12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea0f3f9ce1949beb362a242a84d9400, user_id=747fb356fcab40e2b96926f3a120f022, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:44.649167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9e891-e2ab-4267-a528-e7ae3755fc12 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:44.685778 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1123/2263] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:44 2022] PUT /identity/v3/projects/6ea0f3f9ce1949beb362a242a84d9400/users/747fb356fcab40e2b96926f3a120f022/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:44.690044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23776fdf-6d19-46a0-ac4d-b2739ef38a56 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:44.709776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23776fdf-6d19-46a0-ac4d-b2739ef38a56 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:44.785595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0f77a-7158-4637-a777-5d461c8b4ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:44.786439 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0f77a-7158-4637-a777-5d461c8b4ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:44.786703 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0f77a-7158-4637-a777-5d461c8b4ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:44.788417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0f77a-7158-4637-a777-5d461c8b4ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:44.788417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0f77a-7158-4637-a777-5d461c8b4ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:44.799803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0f77a-7158-4637-a777-5d461c8b4ed6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:44.836644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9e891-e2ab-4267-a528-e7ae3755fc12 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:44.844118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1141/2264] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:19:44.849864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9e891-e2ab-4267-a528-e7ae3755fc12 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:44.878717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f203138-5b30-4638-b20f-eda834ac6c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:44.879779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f203138-5b30-4638-b20f-eda834ac6c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:44.880068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f203138-5b30-4638-b20f-eda834ac6c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:44.880348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f203138-5b30-4638-b20f-eda834ac6c6d admin admin] PATH_INFO: `/v3/projects/6ea0f3f9ce1949beb362a242a84d9400/users/747fb356fcab40e2b96926f3a120f022/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:44.890286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f203138-5b30-4638-b20f-eda834ac6c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea0f3f9ce1949beb362a242a84d9400, user_id=747fb356fcab40e2b96926f3a120f022, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:44.893535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f203138-5b30-4638-b20f-eda834ac6c6d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:44.928558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1124/2265] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:44 2022] PUT /identity/v3/projects/6ea0f3f9ce1949beb362a242a84d9400/users/747fb356fcab40e2b96926f3a120f022/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:46.381178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0f77a-7158-4637-a777-5d461c8b4ed6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:46.396920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0f77a-7158-4637-a777-5d461c8b4ed6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:46.401607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:46.473175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:46.474495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:46.474960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:46.475731 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] PATH_INFO: `/v3/users/617b5182a15945d590058a691d141fe6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:46.477283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=617b5182a15945d590058a691d141fe6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:46.479571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:46.556914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '617b5182a15945d590058a691d141fe6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:46.570883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '617b5182a15945d590058a691d141fe6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:46.572952 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '617b5182a15945d590058a691d141fe6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:46.574067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '617b5182a15945d590058a691d141fe6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:46.593851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '617b5182a15945d590058a691d141fe6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:46.610904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1142/2266] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:46 2022] DELETE /identity/v3/users/617b5182a15945d590058a691d141fe6 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:46.615273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3f833-f3f2-4a8e-b60d-02e6d1ed64c9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:46.615831 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3f833-f3f2-4a8e-b60d-02e6d1ed64c9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:46.615831 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3f833-f3f2-4a8e-b60d-02e6d1ed64c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:46.657916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-68d3f833-f3f2-4a8e-b60d-02e6d1ed64c9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:46.659351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-68d3f833-f3f2-4a8e-b60d-02e6d1ed64c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:46.670621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:46.749259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1125/2267] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:46.757518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:46.759237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:46.843737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc2605-ef06-4416-9deb-4053b2a67a4f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:46.849022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43420ddf-218f-40d6-ad47-b7b66298f12f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:46.849811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43420ddf-218f-40d6-ad47-b7b66298f12f service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:46.850666 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43420ddf-218f-40d6-ad47-b7b66298f12f service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:46.850958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43420ddf-218f-40d6-ad47-b7b66298f12f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:46.867174 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43420ddf-218f-40d6-ad47-b7b66298f12f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:46.870055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43420ddf-218f-40d6-ad47-b7b66298f12f service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:46.899295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1143/2268] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:47.599663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3f833-f3f2-4a8e-b60d-02e6d1ed64c9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:47.601336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-68d3f833-f3f2-4a8e-b60d-02e6d1ed64c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:47.625017 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3f833-f3f2-4a8e-b60d-02e6d1ed64c9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:47.630437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:47.631319 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:47.631604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:47.631874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] PATH_INFO: `/v3/projects/42f3f6fc30e544239e23b4ea6191fe36` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:47.634616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f3f6fc30e544239e23b4ea6191fe36)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:47.637119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:47.779009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f3f6fc30e544239e23b4ea6191fe36', 'request_id': 'req-2326a21a-4559-4747-a300-a0ef615f2705'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:47.793423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f3f6fc30e544239e23b4ea6191fe36', 'request_id': 'req-2326a21a-4559-4747-a300-a0ef615f2705'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:47.825887 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f3f6fc30e544239e23b4ea6191fe36', 'request_id': 'req-2326a21a-4559-4747-a300-a0ef615f2705'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:47.862479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] The token cache is being invalidate because project 42f3f6fc30e544239e23b4ea6191fe36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:47.862780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:47.865761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:47.868047 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1126/2269] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:47 2022] DELETE /identity/v3/projects/42f3f6fc30e544239e23b4ea6191fe36 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:47.871816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43420ddf-218f-40d6-ad47-b7b66298f12f service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:47.882960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43420ddf-218f-40d6-ad47-b7b66298f12f service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:47.946013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:48.054455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:48.055495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:48.056057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:48.056404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] PATH_INFO: `/v3/users/0e63efdebc144e08923c32689e0b5feb` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:48.058009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e63efdebc144e08923c32689e0b5feb)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:48.060524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:48.144180 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e63efdebc144e08923c32689e0b5feb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:48.163351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e63efdebc144e08923c32689e0b5feb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:48.169280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e63efdebc144e08923c32689e0b5feb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:48.170093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e63efdebc144e08923c32689e0b5feb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:48.192369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e63efdebc144e08923c32689e0b5feb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:48.207964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1144/2270] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:47 2022] DELETE /identity/v3/users/0e63efdebc144e08923c32689e0b5feb => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:48.642282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:48.644125 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:48.664390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326a21a-4559-4747-a300-a0ef615f2705 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:48.748075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:48.748939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:48.749219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:48.749490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] PATH_INFO: `/v3/projects/57acd2f223b24dff8ece49b71d1b873b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:48.751168 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] RBAC: Authorizing `identity:delete_project(project_id=57acd2f223b24dff8ece49b71d1b873b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:48.753509 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:48.801359 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:48.808831 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57acd2f223b24dff8ece49b71d1b873b', 'request_id': 'req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:48.823997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57acd2f223b24dff8ece49b71d1b873b', 'request_id': 'req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:48.833717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57acd2f223b24dff8ece49b71d1b873b', 'request_id': 'req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:48.850353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] The token cache is being invalidate because project 57acd2f223b24dff8ece49b71d1b873b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:48.850747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:48.851482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:48.853340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1127/2271] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:48 2022] DELETE /identity/v3/projects/57acd2f223b24dff8ece49b71d1b873b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:48.857077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:48.858719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:48.876822 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a75ee2-81ad-430b-abe7-2bfedfc5fd54 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:48.943257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c608c-df02-4e07-8504-a825dbf845f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:48.944309 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c608c-df02-4e07-8504-a825dbf845f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:48.944695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c608c-df02-4e07-8504-a825dbf845f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:48.945067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c608c-df02-4e07-8504-a825dbf845f2 admin admin] PATH_INFO: `/v3/users/3caeeee0b7ca4f8193483f3f5bbf53ac` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:48.946423 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c608c-df02-4e07-8504-a825dbf845f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3caeeee0b7ca4f8193483f3f5bbf53ac)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:48.949712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c608c-df02-4e07-8504-a825dbf845f2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:49.007362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a50c608c-df02-4e07-8504-a825dbf845f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3caeeee0b7ca4f8193483f3f5bbf53ac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:49.010600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:49.018310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a50c608c-df02-4e07-8504-a825dbf845f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3caeeee0b7ca4f8193483f3f5bbf53ac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:49.018986 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a50c608c-df02-4e07-8504-a825dbf845f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3caeeee0b7ca4f8193483f3f5bbf53ac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:49.019956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a50c608c-df02-4e07-8504-a825dbf845f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3caeeee0b7ca4f8193483f3f5bbf53ac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:49.033758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a50c608c-df02-4e07-8504-a825dbf845f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3caeeee0b7ca4f8193483f3f5bbf53ac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:49.052655 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1145/2272] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:48 2022] DELETE /identity/v3/users/3caeeee0b7ca4f8193483f3f5bbf53ac => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:49.730241 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:49.732204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:49.762722 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4991db8-61a8-4fb8-99b6-8c7eba9fd90d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:49.864275 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:49.865056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:49.865333 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:49.865602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] PATH_INFO: `/v3/projects/31e0b8b8ad704632afe48895c962fea9` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:49.867488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e0b8b8ad704632afe48895c962fea9)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:49.870105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:49.947189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e0b8b8ad704632afe48895c962fea9', 'request_id': 'req-42adcc12-257b-47e0-916f-cfacb9f5bb8d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:49.960895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e0b8b8ad704632afe48895c962fea9', 'request_id': 'req-42adcc12-257b-47e0-916f-cfacb9f5bb8d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:49.972396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e0b8b8ad704632afe48895c962fea9', 'request_id': 'req-42adcc12-257b-47e0-916f-cfacb9f5bb8d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:49.984835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] The token cache is being invalidate because project 31e0b8b8ad704632afe48895c962fea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:49.986235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:49.987415 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:49.989487 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1128/2273] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:49 2022] DELETE /identity/v3/projects/31e0b8b8ad704632afe48895c962fea9 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:51.444954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4b25a-b9b2-48e6-8087-5248b861f56b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:51.445948 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4b25a-b9b2-48e6-8087-5248b861f56b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:51.446432 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4b25a-b9b2-48e6-8087-5248b861f56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:51.477248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c2f4b25a-b9b2-48e6-8087-5248b861f56b None None] MFA Rules not processed for user `747fb356fcab40e2b96926f3a120f022`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:51.478697 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c2f4b25a-b9b2-48e6-8087-5248b861f56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:51.556692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1146/2274] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:51.569249 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:51.570694 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:51.630972 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adcc12-257b-47e0-916f-cfacb9f5bb8d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:51.634665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52736e58-d201-42b8-8df5-90298829eac9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a67120485f74f8bb0ff188cd3abd21c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:51.635321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52736e58-d201-42b8-8df5-90298829eac9 service glance] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:51.635612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52736e58-d201-42b8-8df5-90298829eac9 service glance] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:51.635888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52736e58-d201-42b8-8df5-90298829eac9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:51.637747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-52736e58-d201-42b8-8df5-90298829eac9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:51.652071 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:51.705919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52736e58-d201-42b8-8df5-90298829eac9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:51.707377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52736e58-d201-42b8-8df5-90298829eac9 service glance] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:51.717003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1129/2275] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:51.892736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5162f-342b-4036-8adb-fc044a876df4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:51.893421 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5162f-342b-4036-8adb-fc044a876df4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:51.893844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5162f-342b-4036-8adb-fc044a876df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:51.929750 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8cc5162f-342b-4036-8adb-fc044a876df4 None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:51.931106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8cc5162f-342b-4036-8adb-fc044a876df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:52.032914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1147/2276] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:19:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:52.100950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52736e58-d201-42b8-8df5-90298829eac9 service glance] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:52.103060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-52736e58-d201-42b8-8df5-90298829eac9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:52.207730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52736e58-d201-42b8-8df5-90298829eac9 service glance] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:52.226442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169c7f4-188f-435f-b8c7-b2cdfee37dd0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:52.227426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169c7f4-188f-435f-b8c7-b2cdfee37dd0 service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:52.228031 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169c7f4-188f-435f-b8c7-b2cdfee37dd0 service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:52.229147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169c7f4-188f-435f-b8c7-b2cdfee37dd0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:52.242049 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169c7f4-188f-435f-b8c7-b2cdfee37dd0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:52.243778 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169c7f4-188f-435f-b8c7-b2cdfee37dd0 service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:52.265093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1130/2277] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:19:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:19:55.599541 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5162f-342b-4036-8adb-fc044a876df4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:55.600999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8cc5162f-342b-4036-8adb-fc044a876df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:55.717830 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:55.761103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5162f-342b-4036-8adb-fc044a876df4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:55.766892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:55.768221 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:55.768721 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:55.769190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] PATH_INFO: `/v3/users/a1373aff4fc64b6481a175aff012276c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:55.771353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1373aff4fc64b6481a175aff012276c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:55.774240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:55.872721 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1373aff4fc64b6481a175aff012276c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:55.883817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1373aff4fc64b6481a175aff012276c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:55.885660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1373aff4fc64b6481a175aff012276c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:55.887544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1373aff4fc64b6481a175aff012276c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:55.907352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1373aff4fc64b6481a175aff012276c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:55.924418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1148/2278] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:19:55 2022] DELETE /identity/v3/users/a1373aff4fc64b6481a175aff012276c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:55.928926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750a125-535c-4077-bc6a-09fcdcd5d145 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:55.929876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750a125-535c-4077-bc6a-09fcdcd5d145 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:55.930343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750a125-535c-4077-bc6a-09fcdcd5d145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:55.959452 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a750a125-535c-4077-bc6a-09fcdcd5d145 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:55.961718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a750a125-535c-4077-bc6a-09fcdcd5d145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:56.091367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1131/2279] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:56.103644 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:56.105406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:56.248559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900dbc0-3d0b-4da2-9f9a-7e1670720d39 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:56.256484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516bac8-91e9-4b1d-b4f3-ae366d4307e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:56.257142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516bac8-91e9-4b1d-b4f3-ae366d4307e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:56.257417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516bac8-91e9-4b1d-b4f3-ae366d4307e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:56.257684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516bac8-91e9-4b1d-b4f3-ae366d4307e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:56.291408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516bac8-91e9-4b1d-b4f3-ae366d4307e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:56.292871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516bac8-91e9-4b1d-b4f3-ae366d4307e2 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:56.311717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1149/2280] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:56.857980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750a125-535c-4077-bc6a-09fcdcd5d145 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:56.859758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a750a125-535c-4077-bc6a-09fcdcd5d145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:56.885090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750a125-535c-4077-bc6a-09fcdcd5d145 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:56.893426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4141b-ef6b-4830-9fe4-56815fefb701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:56.894619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4141b-ef6b-4830-9fe4-56815fefb701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:56.895079 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4141b-ef6b-4830-9fe4-56815fefb701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:56.895512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4141b-ef6b-4830-9fe4-56815fefb701 admin admin] PATH_INFO: `/v3/projects/24b643e4d06847748e4090a93c0f0057` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:56.897085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4141b-ef6b-4830-9fe4-56815fefb701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24b643e4d06847748e4090a93c0f0057)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:56.899501 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4141b-ef6b-4830-9fe4-56815fefb701 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:57.027428 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:57.043518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c1c4141b-ef6b-4830-9fe4-56815fefb701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24b643e4d06847748e4090a93c0f0057', 'request_id': 'req-c1c4141b-ef6b-4830-9fe4-56815fefb701'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:57.069151 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c1c4141b-ef6b-4830-9fe4-56815fefb701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24b643e4d06847748e4090a93c0f0057', 'request_id': 'req-c1c4141b-ef6b-4830-9fe4-56815fefb701'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:57.080975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c1c4141b-ef6b-4830-9fe4-56815fefb701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24b643e4d06847748e4090a93c0f0057', 'request_id': 'req-c1c4141b-ef6b-4830-9fe4-56815fefb701'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:57.106826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c1c4141b-ef6b-4830-9fe4-56815fefb701 admin admin] The token cache is being invalidate because project 24b643e4d06847748e4090a93c0f0057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:57.107207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c1c4141b-ef6b-4830-9fe4-56815fefb701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:57.108118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c1c4141b-ef6b-4830-9fe4-56815fefb701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:57.112057 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1132/2281] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:19:56 2022] DELETE /identity/v3/projects/24b643e4d06847748e4090a93c0f0057 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:57.117175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f23c68-c373-46a0-8821-084e2ad7a571 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:57.118296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f23c68-c373-46a0-8821-084e2ad7a571 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:57.118603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f23c68-c373-46a0-8821-084e2ad7a571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:57.153962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-39f23c68-c373-46a0-8821-084e2ad7a571 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:57.155541 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-39f23c68-c373-46a0-8821-084e2ad7a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:57.226670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:57.245944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1150/2282] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:57.256448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972312c-683d-40fb-822e-ede26dda9d58 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:57.257024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972312c-683d-40fb-822e-ede26dda9d58 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:57.257522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972312c-683d-40fb-822e-ede26dda9d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:57.286287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f972312c-683d-40fb-822e-ede26dda9d58 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:57.287974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f972312c-683d-40fb-822e-ede26dda9d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:57.298554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1133/2283] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:57.301770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f23c68-c373-46a0-8821-084e2ad7a571 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:57.312125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f23c68-c373-46a0-8821-084e2ad7a571 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:57.335377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181f129-cfe8-47ff-b472-8c6737ea6d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:57.336201 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181f129-cfe8-47ff-b472-8c6737ea6d1e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:57.336582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181f129-cfe8-47ff-b472-8c6737ea6d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:57.336961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181f129-cfe8-47ff-b472-8c6737ea6d1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:57.337824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181f129-cfe8-47ff-b472-8c6737ea6d1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:19:57.338233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181f129-cfe8-47ff-b472-8c6737ea6d1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:57.339983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181f129-cfe8-47ff-b472-8c6737ea6d1e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:57.357023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1151/2284] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:19:57.357951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972312c-683d-40fb-822e-ede26dda9d58 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:57.369802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972312c-683d-40fb-822e-ede26dda9d58 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:57.375010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ed34c-85c1-4599-8cfb-a92404977bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:57.375721 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ed34c-85c1-4599-8cfb-a92404977bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:57.376137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ed34c-85c1-4599-8cfb-a92404977bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:57.376638 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ed34c-85c1-4599-8cfb-a92404977bd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:57.377344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ed34c-85c1-4599-8cfb-a92404977bd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:57.379273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ed34c-85c1-4599-8cfb-a92404977bd4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:57.413679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1134/2285] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:19:57 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:57.414849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181f129-cfe8-47ff-b472-8c6737ea6d1e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:57.428788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181f129-cfe8-47ff-b472-8c6737ea6d1e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:57.503670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c6e60-c13a-4911-af52-8e3794b89cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:57.504514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c6e60-c13a-4911-af52-8e3794b89cca admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:57.504916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c6e60-c13a-4911-af52-8e3794b89cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:57.505296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c6e60-c13a-4911-af52-8e3794b89cca admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:57.506401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c6e60-c13a-4911-af52-8e3794b89cca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:57.508575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c6e60-c13a-4911-af52-8e3794b89cca admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:57.536449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1152/2286] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:19:57 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:57.537910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ed34c-85c1-4599-8cfb-a92404977bd4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:57.553332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ed34c-85c1-4599-8cfb-a92404977bd4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:57.566843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d0ee3-ee23-493e-af43-ccac4eca3f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:57.567601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d0ee3-ee23-493e-af43-ccac4eca3f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:57.567960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d0ee3-ee23-493e-af43-ccac4eca3f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:57.568302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d0ee3-ee23-493e-af43-ccac4eca3f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:57.569163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d0ee3-ee23-493e-af43-ccac4eca3f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:57.571474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d0ee3-ee23-493e-af43-ccac4eca3f89 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:57.595499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1135/2287] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:19:57.597097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c6e60-c13a-4911-af52-8e3794b89cca admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:57.609847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c6e60-c13a-4911-af52-8e3794b89cca admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:57.614867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228c36a-c8a3-439b-b9fa-4b909bc878d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:57.615653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228c36a-c8a3-439b-b9fa-4b909bc878d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:57.615944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228c36a-c8a3-439b-b9fa-4b909bc878d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:57.616230 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228c36a-c8a3-439b-b9fa-4b909bc878d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:57.617098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228c36a-c8a3-439b-b9fa-4b909bc878d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:57.619013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228c36a-c8a3-439b-b9fa-4b909bc878d7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:57.636185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1153/2288] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:57.637541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d0ee3-ee23-493e-af43-ccac4eca3f89 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:57.650418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d0ee3-ee23-493e-af43-ccac4eca3f89 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:57.663401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78494f1-3c45-4124-bd37-307e6c258a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:57.664592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78494f1-3c45-4124-bd37-307e6c258a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:57.665039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78494f1-3c45-4124-bd37-307e6c258a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:57.665314 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78494f1-3c45-4124-bd37-307e6c258a86 admin admin] PATH_INFO: `/v3/projects/73e8074335b04d60987b4f8b1371a9bb/users/ba793692ba62427a86d52880f1f659a8/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:57.685166 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78494f1-3c45-4124-bd37-307e6c258a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e8074335b04d60987b4f8b1371a9bb, user_id=ba793692ba62427a86d52880f1f659a8, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:57.688707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78494f1-3c45-4124-bd37-307e6c258a86 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:57.707947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1136/2289] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:19:57 2022] PUT /identity/v3/projects/73e8074335b04d60987b4f8b1371a9bb/users/ba793692ba62427a86d52880f1f659a8/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:57.718474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861cb82-810e-41fd-8eeb-4e6058a2c1c7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:57.719420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861cb82-810e-41fd-8eeb-4e6058a2c1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:57.720093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861cb82-810e-41fd-8eeb-4e6058a2c1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:57.755049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4861cb82-810e-41fd-8eeb-4e6058a2c1c7 None None] MFA Rules not processed for user `ba793692ba62427a86d52880f1f659a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:57.763951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4861cb82-810e-41fd-8eeb-4e6058a2c1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:57.842604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1154/2290] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:57.853586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78494f1-3c45-4124-bd37-307e6c258a86 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:57.857089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b78494f1-3c45-4124-bd37-307e6c258a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:57.955388 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78494f1-3c45-4124-bd37-307e6c258a86 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:57.959745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093ed9a-7f2e-4c87-bf8a-44c1352af781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:57.960587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ed9a-7f2e-4c87-bf8a-44c1352af781 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:57.961035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ed9a-7f2e-4c87-bf8a-44c1352af781 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:57.961560 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ed9a-7f2e-4c87-bf8a-44c1352af781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:57.977998 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093ed9a-7f2e-4c87-bf8a-44c1352af781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:57.979842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093ed9a-7f2e-4c87-bf8a-44c1352af781 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:57.995265 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1137/2291] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:19:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:59.601531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53879700-0ef5-4c1e-9546-f73cd5855c87 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:59.602107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53879700-0ef5-4c1e-9546-f73cd5855c87 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:59.602107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53879700-0ef5-4c1e-9546-f73cd5855c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:59.621656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-53879700-0ef5-4c1e-9546-f73cd5855c87 None None] MFA Rules not processed for user `ba793692ba62427a86d52880f1f659a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:59.622911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-53879700-0ef5-4c1e-9546-f73cd5855c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:59.630387 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1155/2292] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:19:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:20:03.381554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71047f2f-bed3-46f5-b24d-ddabd5f94461 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:03.382202 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71047f2f-bed3-46f5-b24d-ddabd5f94461 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:03.382202 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71047f2f-bed3-46f5-b24d-ddabd5f94461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:03.410473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-71047f2f-bed3-46f5-b24d-ddabd5f94461 None None] MFA Rules not processed for user `3b3f8de4a3b54d58806506b827cfa89a`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:20:03.411685 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-71047f2f-bed3-46f5-b24d-ddabd5f94461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:03.486227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1138/2293] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:20:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:20:04.258744 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08d303-2608-4a1b-a089-2af55e2e1e1e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:04.259291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08d303-2608-4a1b-a089-2af55e2e1e1e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:04.259291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08d303-2608-4a1b-a089-2af55e2e1e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:04.283258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9b08d303-2608-4a1b-a089-2af55e2e1e1e None None] MFA Rules not processed for user `5462b6a8d72749b9a07cd52ff8cb29d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:20:04.284560 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9b08d303-2608-4a1b-a089-2af55e2e1e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:04.359462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:04.379692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1156/2294] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:20:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:20:08.458023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8774e-9bc7-40a2-9c89-236146e9396e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:08.458537 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8774e-9bc7-40a2-9c89-236146e9396e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:08.458633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8774e-9bc7-40a2-9c89-236146e9396e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:08.479881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c0d8774e-9bc7-40a2-9c89-236146e9396e None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:20:08.481229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c0d8774e-9bc7-40a2-9c89-236146e9396e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:08.488010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:08.556282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1139/2295] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:20:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:20:08.583398 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08d303-2608-4a1b-a089-2af55e2e1e1e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:08.585141 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9b08d303-2608-4a1b-a089-2af55e2e1e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:08.686572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08d303-2608-4a1b-a089-2af55e2e1e1e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:08.701213 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1393ca-551c-4493-9459-78bc38deb5c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:08.701962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1393ca-551c-4493-9459-78bc38deb5c8 service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:08.702242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1393ca-551c-4493-9459-78bc38deb5c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:08.703112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1393ca-551c-4493-9459-78bc38deb5c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:08.718793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1393ca-551c-4493-9459-78bc38deb5c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:08.721396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1393ca-551c-4493-9459-78bc38deb5c8 service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:08.736425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1157/2296] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:20:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:20:18.545780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=20 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:18.731862 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=37 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:38.765429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8774e-9bc7-40a2-9c89-236146e9396e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:38.767345 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c0d8774e-9bc7-40a2-9c89-236146e9396e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:38.827372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8774e-9bc7-40a2-9c89-236146e9396e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:38.831857 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccab06-c926-44f2-b838-db443b0e2a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:38.832498 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccab06-c926-44f2-b838-db443b0e2a38 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:38.832779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccab06-c926-44f2-b838-db443b0e2a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:38.833048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccab06-c926-44f2-b838-db443b0e2a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:38.834864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-73ccab06-c926-44f2-b838-db443b0e2a38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:38.898245 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccab06-c926-44f2-b838-db443b0e2a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:38.899672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccab06-c926-44f2-b838-db443b0e2a38 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:38.909404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1140/2297] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:20:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:20:40.367333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea4d29-f34e-498e-96a7-101945620846 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:40.367841 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea4d29-f34e-498e-96a7-101945620846 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:40.368107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea4d29-f34e-498e-96a7-101945620846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:40.418979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ccea4d29-f34e-498e-96a7-101945620846 None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:20:40.420219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ccea4d29-f34e-498e-96a7-101945620846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:40.438557 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1158/2298] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:20:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:20:40.450629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccab06-c926-44f2-b838-db443b0e2a38 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:40.471710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccab06-c926-44f2-b838-db443b0e2a38 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:40.476821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c97e0-858d-4432-b1eb-3382962e54ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:40.477567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c97e0-858d-4432-b1eb-3382962e54ec service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:40.477881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c97e0-858d-4432-b1eb-3382962e54ec service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:40.478155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c97e0-858d-4432-b1eb-3382962e54ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:40.487665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c97e0-858d-4432-b1eb-3382962e54ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:40.489342 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c97e0-858d-4432-b1eb-3382962e54ec service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:40.503881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1141/2299] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:20:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:20:41.270094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a44ffa-ed71-46c9-971b-9f3e89c854bb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:41.270434 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a44ffa-ed71-46c9-971b-9f3e89c854bb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:41.270709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a44ffa-ed71-46c9-971b-9f3e89c854bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:41.303045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-72a44ffa-ed71-46c9-971b-9f3e89c854bb None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:20:41.305767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-72a44ffa-ed71-46c9-971b-9f3e89c854bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:41.317286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1159/2300] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:20:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:20:41.335876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c97e0-858d-4432-b1eb-3382962e54ec service swift] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:41.355726 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ced5d-771f-46b8-9ce2-c43f11cc63b7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:41.356551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ced5d-771f-46b8-9ce2-c43f11cc63b7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:41.357194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ced5d-771f-46b8-9ce2-c43f11cc63b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:41.357344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c97e0-858d-4432-b1eb-3382962e54ec service swift] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:41.368802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7992a0-7963-4c0e-9a47-ac3e61c08718 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:41.369777 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7992a0-7963-4c0e-9a47-ac3e61c08718 service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:41.370212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7992a0-7963-4c0e-9a47-ac3e61c08718 service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:41.370648 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7992a0-7963-4c0e-9a47-ac3e61c08718 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:41.386666 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c59ced5d-771f-46b8-9ce2-c43f11cc63b7 None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:20:41.389913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c59ced5d-771f-46b8-9ce2-c43f11cc63b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:41.394158 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7992a0-7963-4c0e-9a47-ac3e61c08718 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:41.395970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7992a0-7963-4c0e-9a47-ac3e61c08718 service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:41.409931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1160/2301] 10.222.0.46 () {58 vars in 967 bytes} [Thu May 26 18:20:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:20:41.418125 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1142/2302] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:20:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:20:41.427047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ced5d-771f-46b8-9ce2-c43f11cc63b7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:41.446105 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ced5d-771f-46b8-9ce2-c43f11cc63b7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:41.460815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d256e-9f53-4b48-8aea-0add5e5cf2c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:41.462324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d256e-9f53-4b48-8aea-0add5e5cf2c4 service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:41.462824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d256e-9f53-4b48-8aea-0add5e5cf2c4 service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:41.463251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d256e-9f53-4b48-8aea-0add5e5cf2c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:41.486217 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d256e-9f53-4b48-8aea-0add5e5cf2c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:41.488940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d256e-9f53-4b48-8aea-0add5e5cf2c4 service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:41.512694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1161/2303] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:20:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:20:42.082105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5940c6a-8657-4c5d-9df1-20059a902c5c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:42.082788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5940c6a-8657-4c5d-9df1-20059a902c5c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:42.083133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5940c6a-8657-4c5d-9df1-20059a902c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:42.110945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c5940c6a-8657-4c5d-9df1-20059a902c5c None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:20:42.113343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c5940c6a-8657-4c5d-9df1-20059a902c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:42.124300 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1143/2304] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:20:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:20:42.141119 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d256e-9f53-4b48-8aea-0add5e5cf2c4 service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:42.155567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d256e-9f53-4b48-8aea-0add5e5cf2c4 service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:42.164018 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaa411-7718-4acd-aa2f-252480d88217 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:42.164692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaa411-7718-4acd-aa2f-252480d88217 service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:42.164974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaa411-7718-4acd-aa2f-252480d88217 service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:42.165248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaa411-7718-4acd-aa2f-252480d88217 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:42.191473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eaa411-7718-4acd-aa2f-252480d88217 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:42.193402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eaa411-7718-4acd-aa2f-252480d88217 service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:42.214967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1162/2305] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:20:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:20:42.880104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94940657-94e5-4e56-96be-e5ca6f1b812c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:42.880484 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94940657-94e5-4e56-96be-e5ca6f1b812c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:42.880833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94940657-94e5-4e56-96be-e5ca6f1b812c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:42.914654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-94940657-94e5-4e56-96be-e5ca6f1b812c None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:20:42.916508 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-94940657-94e5-4e56-96be-e5ca6f1b812c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:42.925645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1144/2306] 10.222.0.46 () {58 vars in 967 bytes} [Thu May 26 18:20:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:20:42.951869 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaa411-7718-4acd-aa2f-252480d88217 service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:42.964549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaa411-7718-4acd-aa2f-252480d88217 service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:42.972731 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9183c-ee58-415f-8516-16dc56013409 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:42.973890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9183c-ee58-415f-8516-16dc56013409 service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:42.974324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9183c-ee58-415f-8516-16dc56013409 service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:42.974801 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9183c-ee58-415f-8516-16dc56013409 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:42.985041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9183c-ee58-415f-8516-16dc56013409 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:42.987276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9183c-ee58-415f-8516-16dc56013409 service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:43.002565 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1163/2307] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:20:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:20:43.974859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94940657-94e5-4e56-96be-e5ca6f1b812c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:43.984754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-94940657-94e5-4e56-96be-e5ca6f1b812c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:44.200118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:44.293641 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94940657-94e5-4e56-96be-e5ca6f1b812c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:44.323524 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995d839-77e1-4558-8055-0c898dcdf4e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:44.324413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995d839-77e1-4558-8055-0c898dcdf4e2 service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:44.325200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995d839-77e1-4558-8055-0c898dcdf4e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:44.326427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995d839-77e1-4558-8055-0c898dcdf4e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:44.333875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5995d839-77e1-4558-8055-0c898dcdf4e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:44.378497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995d839-77e1-4558-8055-0c898dcdf4e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:44.385283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995d839-77e1-4558-8055-0c898dcdf4e2 service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:44.408463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1145/2308] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:20:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:20:46.095334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9183c-ee58-415f-8516-16dc56013409 service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:46.180762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9183c-ee58-415f-8516-16dc56013409 service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:46.203684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:46.204863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:46.205304 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:46.205762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] PATH_INFO: `/v3/users/ba793692ba62427a86d52880f1f659a8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:46.207662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba793692ba62427a86d52880f1f659a8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:46.209932 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:46.283209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba793692ba62427a86d52880f1f659a8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:20:46.305482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba793692ba62427a86d52880f1f659a8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:20:46.306684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba793692ba62427a86d52880f1f659a8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:20:46.307995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba793692ba62427a86d52880f1f659a8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:20:46.332309 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:46.336354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba793692ba62427a86d52880f1f659a8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:20:46.356490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1164/2309] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:20:46 2022] DELETE /identity/v3/users/ba793692ba62427a86d52880f1f659a8 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:20:46.358483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f042a-61c3-4fcb-b748-162fc5fd21a1 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:46.359347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f042a-61c3-4fcb-b748-162fc5fd21a1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:46.359676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f042a-61c3-4fcb-b748-162fc5fd21a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:46.383995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d35f042a-61c3-4fcb-b748-162fc5fd21a1 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:20:46.385361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d35f042a-61c3-4fcb-b748-162fc5fd21a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:46.459420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1146/2310] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:20:46.466139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:46.468152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:46.549807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004d456-ff8e-48de-8235-2501ce3056ed admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:46.553488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6787c-2067-4e7c-9c0f-9cfa2c0c8e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:46.554194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6787c-2067-4e7c-9c0f-9cfa2c0c8e8e service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:46.555328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6787c-2067-4e7c-9c0f-9cfa2c0c8e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:46.555614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6787c-2067-4e7c-9c0f-9cfa2c0c8e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:46.569880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6787c-2067-4e7c-9c0f-9cfa2c0c8e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:46.571335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6787c-2067-4e7c-9c0f-9cfa2c0c8e8e service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:46.594857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1165/2311] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:20:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:20:46.847010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f042a-61c3-4fcb-b748-162fc5fd21a1 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:46.850823 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d35f042a-61c3-4fcb-b748-162fc5fd21a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:46.888543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f042a-61c3-4fcb-b748-162fc5fd21a1 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:46.899235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185592c7-b4ca-43e6-977c-d547e83529fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:46.900328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185592c7-b4ca-43e6-977c-d547e83529fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:46.900807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185592c7-b4ca-43e6-977c-d547e83529fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:46.901280 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185592c7-b4ca-43e6-977c-d547e83529fb admin admin] PATH_INFO: `/v3/projects/73e8074335b04d60987b4f8b1371a9bb` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:46.902752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185592c7-b4ca-43e6-977c-d547e83529fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e8074335b04d60987b4f8b1371a9bb)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:46.905149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185592c7-b4ca-43e6-977c-d547e83529fb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:46.968315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-185592c7-b4ca-43e6-977c-d547e83529fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e8074335b04d60987b4f8b1371a9bb', 'request_id': 'req-185592c7-b4ca-43e6-977c-d547e83529fb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:20:46.976794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-185592c7-b4ca-43e6-977c-d547e83529fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e8074335b04d60987b4f8b1371a9bb', 'request_id': 'req-185592c7-b4ca-43e6-977c-d547e83529fb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:20:46.986001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-185592c7-b4ca-43e6-977c-d547e83529fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e8074335b04d60987b4f8b1371a9bb', 'request_id': 'req-185592c7-b4ca-43e6-977c-d547e83529fb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:20:47.000834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-185592c7-b4ca-43e6-977c-d547e83529fb admin admin] The token cache is being invalidate because project 73e8074335b04d60987b4f8b1371a9bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:20:47.001139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-185592c7-b4ca-43e6-977c-d547e83529fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:20:47.004108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-185592c7-b4ca-43e6-977c-d547e83529fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:20:47.004327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1147/2312] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:20:46 2022] DELETE /identity/v3/projects/73e8074335b04d60987b4f8b1371a9bb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:20:47.009484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a72e1a-b253-4923-839a-84d212e42ada None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:47.009823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a72e1a-b253-4923-839a-84d212e42ada None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:47.010082 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a72e1a-b253-4923-839a-84d212e42ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:47.031980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-55a72e1a-b253-4923-839a-84d212e42ada None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:20:47.033205 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-55a72e1a-b253-4923-839a-84d212e42ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:47.110498 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1166/2313] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:20:47.119330 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98c22f-3550-4b81-a9ad-ab2d77066d9d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:47.119686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98c22f-3550-4b81-a9ad-ab2d77066d9d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:47.120033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98c22f-3550-4b81-a9ad-ab2d77066d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:47.144490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-db98c22f-3550-4b81-a9ad-ab2d77066d9d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:20:47.146064 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-db98c22f-3550-4b81-a9ad-ab2d77066d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:47.160043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1148/2314] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:20:47.164279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a72e1a-b253-4923-839a-84d212e42ada None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:47.175788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a72e1a-b253-4923-839a-84d212e42ada None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:47.180490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c72775-242c-4c9d-bf9e-ac19a1772254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:47.181155 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c72775-242c-4c9d-bf9e-ac19a1772254 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:47.181437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c72775-242c-4c9d-bf9e-ac19a1772254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:47.181713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c72775-242c-4c9d-bf9e-ac19a1772254 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:47.182474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c72775-242c-4c9d-bf9e-ac19a1772254 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:20:47.182786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c72775-242c-4c9d-bf9e-ac19a1772254 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:47.184408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c72775-242c-4c9d-bf9e-ac19a1772254 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:47.201042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1167/2315] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:20:47.202449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98c22f-3550-4b81-a9ad-ab2d77066d9d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:47.212948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98c22f-3550-4b81-a9ad-ab2d77066d9d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:47.217683 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756d56a-e178-4f2a-add5-5ea482e6f65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:47.218552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d56a-e178-4f2a-add5-5ea482e6f65d admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:47.220228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d56a-e178-4f2a-add5-5ea482e6f65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:47.220527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d56a-e178-4f2a-add5-5ea482e6f65d admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:47.221210 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756d56a-e178-4f2a-add5-5ea482e6f65d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:47.223162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756d56a-e178-4f2a-add5-5ea482e6f65d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:47.259909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1149/2316] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:20:47 2022] POST /identity/v3/projects => generated 391 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:20:47.261038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c72775-242c-4c9d-bf9e-ac19a1772254 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:47.277589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c72775-242c-4c9d-bf9e-ac19a1772254 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:47.350899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfe3cf-6c59-440f-a4bd-6501cbcfce51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:47.351628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfe3cf-6c59-440f-a4bd-6501cbcfce51 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:47.351914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfe3cf-6c59-440f-a4bd-6501cbcfce51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:47.352192 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfe3cf-6c59-440f-a4bd-6501cbcfce51 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:47.353072 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfe3cf-6c59-440f-a4bd-6501cbcfce51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:47.355252 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfe3cf-6c59-440f-a4bd-6501cbcfce51 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:47.382920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1168/2317] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:20:47 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:20:47.386611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756d56a-e178-4f2a-add5-5ea482e6f65d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:47.398114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756d56a-e178-4f2a-add5-5ea482e6f65d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:47.403291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e639f-276d-41a7-81cb-461a696a6966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:47.404273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e639f-276d-41a7-81cb-461a696a6966 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:47.404637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e639f-276d-41a7-81cb-461a696a6966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:47.405003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e639f-276d-41a7-81cb-461a696a6966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:47.405860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e639f-276d-41a7-81cb-461a696a6966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:47.407668 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e639f-276d-41a7-81cb-461a696a6966 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:47.425681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1150/2318] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:20:47.430055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfe3cf-6c59-440f-a4bd-6501cbcfce51 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:47.441274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfe3cf-6c59-440f-a4bd-6501cbcfce51 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:47.446017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7b5be-84db-4a01-a43b-7aff32ce3040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:47.446779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7b5be-84db-4a01-a43b-7aff32ce3040 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:47.447128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7b5be-84db-4a01-a43b-7aff32ce3040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:47.447466 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7b5be-84db-4a01-a43b-7aff32ce3040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:47.448245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7b5be-84db-4a01-a43b-7aff32ce3040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:47.449995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7b5be-84db-4a01-a43b-7aff32ce3040 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:47.451883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:47.468137 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1169/2319] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:20:47.471386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e639f-276d-41a7-81cb-461a696a6966 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:47.481229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e639f-276d-41a7-81cb-461a696a6966 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:47.486691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644062e0-3f93-404a-b17a-c0fd29e0a86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:47.487677 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644062e0-3f93-404a-b17a-c0fd29e0a86e admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:47.488033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644062e0-3f93-404a-b17a-c0fd29e0a86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:47.488383 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644062e0-3f93-404a-b17a-c0fd29e0a86e admin admin] PATH_INFO: `/v3/projects/a66e69ae736344d7954a63842b16dca5/users/7a39eae957ed4197934ba3dc40f1dab7/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:47.503988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:47.508407 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644062e0-3f93-404a-b17a-c0fd29e0a86e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66e69ae736344d7954a63842b16dca5, user_id=7a39eae957ed4197934ba3dc40f1dab7, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:47.512041 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644062e0-3f93-404a-b17a-c0fd29e0a86e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:47.531048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1151/2320] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:20:47 2022] PUT /identity/v3/projects/a66e69ae736344d7954a63842b16dca5/users/7a39eae957ed4197934ba3dc40f1dab7/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:20:47.541747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc784d1-8901-4a09-830a-50c1598c73fa None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:47.542069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc784d1-8901-4a09-830a-50c1598c73fa None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:47.542359 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc784d1-8901-4a09-830a-50c1598c73fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:47.575725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-efc784d1-8901-4a09-830a-50c1598c73fa None None] MFA Rules not processed for user `7a39eae957ed4197934ba3dc40f1dab7`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:20:47.575725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-efc784d1-8901-4a09-830a-50c1598c73fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:47.650429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1170/2321] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:20:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:20:47.663787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644062e0-3f93-404a-b17a-c0fd29e0a86e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:47.666075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-644062e0-3f93-404a-b17a-c0fd29e0a86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:47.760154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644062e0-3f93-404a-b17a-c0fd29e0a86e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:47.764397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096e319-808e-4ea5-b576-1c9d15278bbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:47.764969 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096e319-808e-4ea5-b576-1c9d15278bbf service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:47.765267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096e319-808e-4ea5-b576-1c9d15278bbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:47.765658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096e319-808e-4ea5-b576-1c9d15278bbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:47.776901 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096e319-808e-4ea5-b576-1c9d15278bbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:47.778481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096e319-808e-4ea5-b576-1c9d15278bbf service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:47.788677 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1152/2322] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:20:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:20:57.647160 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=34 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:57.783998 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:21:04.779621 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59979cbe-2d44-4866-8ccd-424ec647653e None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:04.780362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59979cbe-2d44-4866-8ccd-424ec647653e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:04.781810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59979cbe-2d44-4866-8ccd-424ec647653e None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:04.784068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1171/2323] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:21:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:21:04.794999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0be97-649f-4994-bc67-9ba230e0b241 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:04.795552 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0be97-649f-4994-bc67-9ba230e0b241 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:04.795961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0be97-649f-4994-bc67-9ba230e0b241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:04.870825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-90c0be97-649f-4994-bc67-9ba230e0b241 None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:21:04.871967 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-90c0be97-649f-4994-bc67-9ba230e0b241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:04.961856 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1153/2324] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:21:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:21:04.979100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59979cbe-2d44-4866-8ccd-424ec647653e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:21:04.981249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-59979cbe-2d44-4866-8ccd-424ec647653e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:05.087907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59979cbe-2d44-4866-8ccd-424ec647653e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:21:05.092167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d0337-3c06-4fa0-b12b-365038219936 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a67120485f74f8bb0ff188cd3abd21c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:21:05.092819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d0337-3c06-4fa0-b12b-365038219936 service glance] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:05.093095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d0337-3c06-4fa0-b12b-365038219936 service glance] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:05.093428 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d0337-3c06-4fa0-b12b-365038219936 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:05.108782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d0337-3c06-4fa0-b12b-365038219936 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:21:05.110409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d0337-3c06-4fa0-b12b-365038219936 service glance] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:21:05.121959 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1172/2325] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:21:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:21:05.192677 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a5a1-cfcc-466b-b063-78cbad444dc5 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:05.193112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a5a1-cfcc-466b-b063-78cbad444dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:05.193435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a5a1-cfcc-466b-b063-78cbad444dc5 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:05.195067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1154/2326] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:21:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:21:05.199886 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa2fed-026a-40c2-8b8b-10ffc8ded620 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:05.200172 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa2fed-026a-40c2-8b8b-10ffc8ded620 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:05.200451 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa2fed-026a-40c2-8b8b-10ffc8ded620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:05.220555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-98aa2fed-026a-40c2-8b8b-10ffc8ded620 None None] MFA Rules not processed for user `f668c4c6745e4f1eae20d8a3a12cf0cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:21:05.221803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-98aa2fed-026a-40c2-8b8b-10ffc8ded620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:05.278903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1173/2327] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:21:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:21:05.291740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a5a1-cfcc-466b-b063-78cbad444dc5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:21:05.293540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4e50a5a1-cfcc-466b-b063-78cbad444dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:05.306944 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a5a1-cfcc-466b-b063-78cbad444dc5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:21:05.310479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb29fc5-e487-417c-8e12-d65be55bee83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:21:05.311108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb29fc5-e487-417c-8e12-d65be55bee83 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:05.311389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb29fc5-e487-417c-8e12-d65be55bee83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:05.311661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb29fc5-e487-417c-8e12-d65be55bee83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:05.318939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb29fc5-e487-417c-8e12-d65be55bee83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:21:05.320367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb29fc5-e487-417c-8e12-d65be55bee83 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:21:05.328720 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1155/2328] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:21:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:21:05.412739 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16cb7d-2727-4933-9ae0-9d444a0c237f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:05.413053 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16cb7d-2727-4933-9ae0-9d444a0c237f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:05.413351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16cb7d-2727-4933-9ae0-9d444a0c237f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:05.414792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1174/2329] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:21:05.419523 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516306c-8eca-411e-91e2-97d5c8137036 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:05.419817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516306c-8eca-411e-91e2-97d5c8137036 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:05.420090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516306c-8eca-411e-91e2-97d5c8137036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:05.439392 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f516306c-8eca-411e-91e2-97d5c8137036 None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:21:05.440596 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f516306c-8eca-411e-91e2-97d5c8137036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:05.450139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1156/2330] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:21:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:21:05.461322 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16cb7d-2727-4933-9ae0-9d444a0c237f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:21:05.463021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2f16cb7d-2727-4933-9ae0-9d444a0c237f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:05.476452 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16cb7d-2727-4933-9ae0-9d444a0c237f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:21:05.480010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d377841-c58a-4680-8c5a-e21f4c08bbde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:21:05.480646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377841-c58a-4680-8c5a-e21f4c08bbde service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:05.480927 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377841-c58a-4680-8c5a-e21f4c08bbde service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:05.481220 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377841-c58a-4680-8c5a-e21f4c08bbde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:05.491811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d377841-c58a-4680-8c5a-e21f4c08bbde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:21:05.493262 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d377841-c58a-4680-8c5a-e21f4c08bbde service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:21:05.505599 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1175/2331] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:21:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:21:07.282312 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516306c-8eca-411e-91e2-97d5c8137036 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:21:07.345578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516306c-8eca-411e-91e2-97d5c8137036 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:21:07.350298 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:21:07.351171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:07.351427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:07.351698 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] PATH_INFO: `/v3/users/7a39eae957ed4197934ba3dc40f1dab7` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:07.352852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a39eae957ed4197934ba3dc40f1dab7)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:21:07.354801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:21:07.395496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:21:07.400839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a39eae957ed4197934ba3dc40f1dab7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:21:07.408182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a39eae957ed4197934ba3dc40f1dab7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:21:07.419772 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a39eae957ed4197934ba3dc40f1dab7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:21:07.420400 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a39eae957ed4197934ba3dc40f1dab7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:21:07.428163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a39eae957ed4197934ba3dc40f1dab7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:21:07.433978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1157/2332] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:21:07 2022] DELETE /identity/v3/users/7a39eae957ed4197934ba3dc40f1dab7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:21:07.438591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092e820-aaba-4527-b574-bdbe03c789cc None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:07.439002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092e820-aaba-4527-b574-bdbe03c789cc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:07.439100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092e820-aaba-4527-b574-bdbe03c789cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:07.457043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e092e820-aaba-4527-b574-bdbe03c789cc None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:21:07.458271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e092e820-aaba-4527-b574-bdbe03c789cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:07.519682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1176/2333] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:21:07.529597 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:21:07.531407 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:07.590400 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59cab05-1da7-4158-820d-a298e8abc229 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:21:07.593985 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464c399-03ad-4145-8f78-2ee67e9d5fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:21:07.594640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464c399-03ad-4145-8f78-2ee67e9d5fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:07.594906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464c399-03ad-4145-8f78-2ee67e9d5fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:07.595179 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464c399-03ad-4145-8f78-2ee67e9d5fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:07.605062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464c399-03ad-4145-8f78-2ee67e9d5fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:21:07.606522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464c399-03ad-4145-8f78-2ee67e9d5fe3 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:21:07.616637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1158/2334] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:21:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:21:07.883090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092e820-aaba-4527-b574-bdbe03c789cc None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:21:07.884898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e092e820-aaba-4527-b574-bdbe03c789cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:07.897570 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092e820-aaba-4527-b574-bdbe03c789cc None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:21:07.902792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:21:07.903962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:07.904239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:07.904511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] PATH_INFO: `/v3/projects/a66e69ae736344d7954a63842b16dca5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:07.905797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a66e69ae736344d7954a63842b16dca5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:21:07.908150 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:21:07.926867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:21:07.961301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a66e69ae736344d7954a63842b16dca5', 'request_id': 'req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:21:07.968017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a66e69ae736344d7954a63842b16dca5', 'request_id': 'req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:21:07.976654 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a66e69ae736344d7954a63842b16dca5', 'request_id': 'req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:21:07.990437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] The token cache is being invalidate because project a66e69ae736344d7954a63842b16dca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:21:07.990719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:21:07.991347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:21:07.992879 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1177/2335] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:21:07 2022] DELETE /identity/v3/projects/a66e69ae736344d7954a63842b16dca5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:21:09.912290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758d060-b69f-455d-b812-3401e2e3f7ac None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:09.912767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758d060-b69f-455d-b812-3401e2e3f7ac None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:09.913038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758d060-b69f-455d-b812-3401e2e3f7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:09.930477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0758d060-b69f-455d-b812-3401e2e3f7ac None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:21:09.931689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0758d060-b69f-455d-b812-3401e2e3f7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:09.991718 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1159/2336] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:21:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:21:10.005582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:21:10.007120 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:10.182242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2f5e4-98ad-4f13-bd43-6d116cc00f44 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:21:10.188400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbfab2-1465-44ad-9906-1da0c20070c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:21:10.189208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbfab2-1465-44ad-9906-1da0c20070c8 service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:10.190406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbfab2-1465-44ad-9906-1da0c20070c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:10.191356 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbfab2-1465-44ad-9906-1da0c20070c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:10.211199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbfab2-1465-44ad-9906-1da0c20070c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:21:10.213404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbfab2-1465-44ad-9906-1da0c20070c8 service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:21:10.225567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1178/2337] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:21:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:21:19.983988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=65,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:21:20.219545 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:21:23.595986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566d883-e7d5-45e5-9805-edc00ddc7df2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:23.596515 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566d883-e7d5-45e5-9805-edc00ddc7df2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:23.596618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566d883-e7d5-45e5-9805-edc00ddc7df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:23.615526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4566d883-e7d5-45e5-9805-edc00ddc7df2 None None] MFA Rules not processed for user `8da0a5a09a6041d68c5e445af1a492f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:21:23.616681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4566d883-e7d5-45e5-9805-edc00ddc7df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:23.678727 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1160/2338] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:21:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:21:33.671105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=25 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:21:34.133657 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef5a45-536e-47e5-94bb-d0b0b2b29684 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:34.134170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef5a45-536e-47e5-94bb-d0b0b2b29684 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:34.134170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef5a45-536e-47e5-94bb-d0b0b2b29684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:34.151783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9aef5a45-536e-47e5-94bb-d0b0b2b29684 None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:21:34.152999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9aef5a45-536e-47e5-94bb-d0b0b2b29684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:34.160443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1179/2339] 10.222.0.46 () {58 vars in 967 bytes} [Thu May 26 18:21:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:21:34.173432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566d883-e7d5-45e5-9805-edc00ddc7df2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:21:34.185555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566d883-e7d5-45e5-9805-edc00ddc7df2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:21:34.190793 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5c8a4-517a-4048-87a2-038ebce93598 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:21:34.191635 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5c8a4-517a-4048-87a2-038ebce93598 service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:34.191921 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5c8a4-517a-4048-87a2-038ebce93598 service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:34.192193 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5c8a4-517a-4048-87a2-038ebce93598 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:34.201070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5c8a4-517a-4048-87a2-038ebce93598 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:21:34.202691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5c8a4-517a-4048-87a2-038ebce93598 service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:21:34.211576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1161/2340] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:21:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:21:44.146352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:21:44.207630 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:21:45.537106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1679e5-007e-4ebc-969d-2ad532bfbb67 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:21:45.537605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1679e5-007e-4ebc-969d-2ad532bfbb67 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:21:45.537719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1679e5-007e-4ebc-969d-2ad532bfbb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:21:45.556860 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6f1679e5-007e-4ebc-969d-2ad532bfbb67 None None] MFA Rules not processed for user `3b3f8de4a3b54d58806506b827cfa89a`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:21:45.558097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6f1679e5-007e-4ebc-969d-2ad532bfbb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:21:45.614306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1180/2341] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:21:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:21:55.607837 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=25 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:22:22.903269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a108117-9aff-4ddc-b60e-3cff283c96f0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:22.903811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a108117-9aff-4ddc-b60e-3cff283c96f0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:22.903896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a108117-9aff-4ddc-b60e-3cff283c96f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:22.922344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5a108117-9aff-4ddc-b60e-3cff283c96f0 None None] MFA Rules not processed for user `7f2ffbc59c60455cb9cadda081e30381`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:22:22.923541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5a108117-9aff-4ddc-b60e-3cff283c96f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:22.980951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1162/2342] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:22:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:22:32.973629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=25 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:22:35.498377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a94dc-9eac-4802-875d-9aa2419aed00 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:35.498901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a94dc-9eac-4802-875d-9aa2419aed00 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:35.499020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a94dc-9eac-4802-875d-9aa2419aed00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:35.517541 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f19a94dc-9eac-4802-875d-9aa2419aed00 None None] MFA Rules not processed for user `a20f6932d763461ea26c2d60ed58a59f`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:22:35.518783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f19a94dc-9eac-4802-875d-9aa2419aed00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:35.580142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1181/2343] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:22:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:22:40.262679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a108117-9aff-4ddc-b60e-3cff283c96f0 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:40.266254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5a108117-9aff-4ddc-b60e-3cff283c96f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:40.365941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a108117-9aff-4ddc-b60e-3cff283c96f0 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:40.368174 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c8aee-2d61-4678-a742-a6847a6a8f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:40.369910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c8aee-2d61-4678-a742-a6847a6a8f24 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:40.370197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c8aee-2d61-4678-a742-a6847a6a8f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:40.371009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c8aee-2d61-4678-a742-a6847a6a8f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:40.374070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-659c8aee-2d61-4678-a742-a6847a6a8f24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:40.570831 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c8aee-2d61-4678-a742-a6847a6a8f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:40.573584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c8aee-2d61-4678-a742-a6847a6a8f24 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:40.606231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1163/2344] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:22:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:22:44.840716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe13d-e906-4106-96d3-5afda7d227c1 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:44.841196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe13d-e906-4106-96d3-5afda7d227c1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:44.841291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe13d-e906-4106-96d3-5afda7d227c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:44.859569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-adbfe13d-e906-4106-96d3-5afda7d227c1 None None] MFA Rules not processed for user `8da0a5a09a6041d68c5e445af1a492f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:22:44.860769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-adbfe13d-e906-4106-96d3-5afda7d227c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:44.868203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1182/2345] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:22:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:22:45.206738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c8aee-2d61-4678-a742-a6847a6a8f24 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:45.216930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c8aee-2d61-4678-a742-a6847a6a8f24 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:45.221495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846e6a9-5cb0-4e1c-b445-c90cea239d7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:45.222262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846e6a9-5cb0-4e1c-b445-c90cea239d7d service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:45.223287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846e6a9-5cb0-4e1c-b445-c90cea239d7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:45.223287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846e6a9-5cb0-4e1c-b445-c90cea239d7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:45.225206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0846e6a9-5cb0-4e1c-b445-c90cea239d7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:45.237828 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846e6a9-5cb0-4e1c-b445-c90cea239d7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:45.239695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846e6a9-5cb0-4e1c-b445-c90cea239d7d service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:45.248654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1164/2346] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:22:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:22:45.689023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfe13d-e906-4106-96d3-5afda7d227c1 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:45.690652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-adbfe13d-e906-4106-96d3-5afda7d227c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:45.693968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846e6a9-5cb0-4e1c-b445-c90cea239d7d service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:45.706069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846e6a9-5cb0-4e1c-b445-c90cea239d7d service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:45.716194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec56eaa-d36a-4bdc-940d-15874da8f4fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:45.717351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec56eaa-d36a-4bdc-940d-15874da8f4fc service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:45.717891 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec56eaa-d36a-4bdc-940d-15874da8f4fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:45.718387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec56eaa-d36a-4bdc-940d-15874da8f4fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:45.722205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9ec56eaa-d36a-4bdc-940d-15874da8f4fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:45.739886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec56eaa-d36a-4bdc-940d-15874da8f4fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:45.742214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec56eaa-d36a-4bdc-940d-15874da8f4fc service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:45.758225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1165/2347] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:22:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:22:45.780425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfe13d-e906-4106-96d3-5afda7d227c1 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:45.786009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e836a2-3430-491c-b9f5-ceb5d1fc1d8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:45.786669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e836a2-3430-491c-b9f5-ceb5d1fc1d8b service placement] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:45.786949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e836a2-3430-491c-b9f5-ceb5d1fc1d8b service placement] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:45.787223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e836a2-3430-491c-b9f5-ceb5d1fc1d8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:45.789013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b9e836a2-3430-491c-b9f5-ceb5d1fc1d8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:45.802606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e836a2-3430-491c-b9f5-ceb5d1fc1d8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:45.804242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e836a2-3430-491c-b9f5-ceb5d1fc1d8b service placement] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:45.816213 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1183/2348] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:22:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:22:50.644944 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec56eaa-d36a-4bdc-940d-15874da8f4fc service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:50.646459 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9ec56eaa-d36a-4bdc-940d-15874da8f4fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:50.660097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec56eaa-d36a-4bdc-940d-15874da8f4fc service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:50.665664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964536fe-d048-4a56-8369-e4e9e4d64749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:50.666291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964536fe-d048-4a56-8369-e4e9e4d64749 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:50.666568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964536fe-d048-4a56-8369-e4e9e4d64749 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:50.666836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964536fe-d048-4a56-8369-e4e9e4d64749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:50.668551 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-964536fe-d048-4a56-8369-e4e9e4d64749 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:50.678990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964536fe-d048-4a56-8369-e4e9e4d64749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:50.681175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964536fe-d048-4a56-8369-e4e9e4d64749 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:50.690451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1166/2349] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:22:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:22:55.811217 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=60 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:22:56.652887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e836a2-3430-491c-b9f5-ceb5d1fc1d8b service placement] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:56.654337 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b9e836a2-3430-491c-b9f5-ceb5d1fc1d8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:56.666171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e836a2-3430-491c-b9f5-ceb5d1fc1d8b service placement] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:56.669642 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b09008-df04-43c0-888c-07f309f19fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:56.670302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b09008-df04-43c0-888c-07f309f19fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:56.670578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b09008-df04-43c0-888c-07f309f19fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:56.670866 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b09008-df04-43c0-888c-07f309f19fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:56.672601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-58b09008-df04-43c0-888c-07f309f19fd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:56.735074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b09008-df04-43c0-888c-07f309f19fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:56.736530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b09008-df04-43c0-888c-07f309f19fd2 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:56.746457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1184/2350] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:22:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:22:57.436727 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d6f65-b64b-4cc3-9e51-c69392a2bc9a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:57.437210 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d6f65-b64b-4cc3-9e51-c69392a2bc9a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:57.437322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d6f65-b64b-4cc3-9e51-c69392a2bc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:57.457324 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-262d6f65-b64b-4cc3-9e51-c69392a2bc9a None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:22:57.458711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-262d6f65-b64b-4cc3-9e51-c69392a2bc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:57.466946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1167/2351] 10.222.0.46 () {58 vars in 967 bytes} [Thu May 26 18:22:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:22:57.516806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b09008-df04-43c0-888c-07f309f19fd2 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:57.528331 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b09008-df04-43c0-888c-07f309f19fd2 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:57.533758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249c319-ff2b-4d66-b69b-4b132528742e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:57.534401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249c319-ff2b-4d66-b69b-4b132528742e service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:57.534686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249c319-ff2b-4d66-b69b-4b132528742e service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:57.534963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249c319-ff2b-4d66-b69b-4b132528742e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:57.543769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249c319-ff2b-4d66-b69b-4b132528742e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:57.545371 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249c319-ff2b-4d66-b69b-4b132528742e service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:57.553814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1185/2352] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:22:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:23:00.676860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d6f65-b64b-4cc3-9e51-c69392a2bc9a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:00.718527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d6f65-b64b-4cc3-9e51-c69392a2bc9a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:00.730840 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692d98b-dffd-4be8-b32b-b0e1abf9d80f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:00.731936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692d98b-dffd-4be8-b32b-b0e1abf9d80f service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:00.732154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692d98b-dffd-4be8-b32b-b0e1abf9d80f service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:00.732425 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692d98b-dffd-4be8-b32b-b0e1abf9d80f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:00.734281 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7692d98b-dffd-4be8-b32b-b0e1abf9d80f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:00.749033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692d98b-dffd-4be8-b32b-b0e1abf9d80f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:00.750705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692d98b-dffd-4be8-b32b-b0e1abf9d80f service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:00.762750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1168/2353] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:00.778876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249c319-ff2b-4d66-b69b-4b132528742e service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:00.780504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6249c319-ff2b-4d66-b69b-4b132528742e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:00.802282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249c319-ff2b-4d66-b69b-4b132528742e service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:00.811000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36aa7a-c6d9-4a80-8db7-a0f39c300901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:00.811889 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36aa7a-c6d9-4a80-8db7-a0f39c300901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:00.812395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36aa7a-c6d9-4a80-8db7-a0f39c300901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:00.812742 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36aa7a-c6d9-4a80-8db7-a0f39c300901 admin admin] PATH_INFO: `/v3/users/3b3f8de4a3b54d58806506b827cfa89a` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:00.814065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36aa7a-c6d9-4a80-8db7-a0f39c300901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b3f8de4a3b54d58806506b827cfa89a)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:00.816243 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36aa7a-c6d9-4a80-8db7-a0f39c300901 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:00.866419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5d36aa7a-c6d9-4a80-8db7-a0f39c300901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b3f8de4a3b54d58806506b827cfa89a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:00.875235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5d36aa7a-c6d9-4a80-8db7-a0f39c300901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b3f8de4a3b54d58806506b827cfa89a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:00.875960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5d36aa7a-c6d9-4a80-8db7-a0f39c300901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b3f8de4a3b54d58806506b827cfa89a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:00.876629 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5d36aa7a-c6d9-4a80-8db7-a0f39c300901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b3f8de4a3b54d58806506b827cfa89a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:00.906928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5d36aa7a-c6d9-4a80-8db7-a0f39c300901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b3f8de4a3b54d58806506b827cfa89a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:00.921160 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1186/2354] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:23:00 2022] DELETE /identity/v3/users/3b3f8de4a3b54d58806506b827cfa89a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:01.118845 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692d98b-dffd-4be8-b32b-b0e1abf9d80f service placement] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:01.130572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692d98b-dffd-4be8-b32b-b0e1abf9d80f service placement] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:01.174961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:01.194413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641109e-ff20-4cf8-b83a-52d9c679f09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:01.195277 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641109e-ff20-4cf8-b83a-52d9c679f09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:01.195554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641109e-ff20-4cf8-b83a-52d9c679f09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:01.195849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641109e-ff20-4cf8-b83a-52d9c679f09b admin admin] PATH_INFO: `/v3/projects/5cb09766b9784d50aad144d466a5d050` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:01.197069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641109e-ff20-4cf8-b83a-52d9c679f09b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cb09766b9784d50aad144d466a5d050)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:01.198999 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641109e-ff20-4cf8-b83a-52d9c679f09b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:01.264197 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f641109e-ff20-4cf8-b83a-52d9c679f09b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cb09766b9784d50aad144d466a5d050', 'request_id': 'req-f641109e-ff20-4cf8-b83a-52d9c679f09b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:01.282529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f641109e-ff20-4cf8-b83a-52d9c679f09b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cb09766b9784d50aad144d466a5d050', 'request_id': 'req-f641109e-ff20-4cf8-b83a-52d9c679f09b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:01.292458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f641109e-ff20-4cf8-b83a-52d9c679f09b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cb09766b9784d50aad144d466a5d050', 'request_id': 'req-f641109e-ff20-4cf8-b83a-52d9c679f09b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:01.306163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f641109e-ff20-4cf8-b83a-52d9c679f09b admin admin] The token cache is being invalidate because project 5cb09766b9784d50aad144d466a5d050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:23:01.306531 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f641109e-ff20-4cf8-b83a-52d9c679f09b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:01.307302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f641109e-ff20-4cf8-b83a-52d9c679f09b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:01.309648 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1169/2355] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:23:01 2022] DELETE /identity/v3/projects/5cb09766b9784d50aad144d466a5d050 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:01.315634 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24271c0f-6c7f-4ccf-9632-82a6bb184522 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:01.316480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24271c0f-6c7f-4ccf-9632-82a6bb184522 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:01.316967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24271c0f-6c7f-4ccf-9632-82a6bb184522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:01.340980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-24271c0f-6c7f-4ccf-9632-82a6bb184522 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:01.342448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-24271c0f-6c7f-4ccf-9632-82a6bb184522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:01.406984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:01.429521 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1187/2356] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:01.437786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888f049-75e7-42f8-ae16-6251ad4b20c8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:01.438082 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888f049-75e7-42f8-ae16-6251ad4b20c8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:01.438365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888f049-75e7-42f8-ae16-6251ad4b20c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:01.465249 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2888f049-75e7-42f8-ae16-6251ad4b20c8 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:01.466639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2888f049-75e7-42f8-ae16-6251ad4b20c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:01.477829 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1170/2357] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:01.481135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24271c0f-6c7f-4ccf-9632-82a6bb184522 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:01.491518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24271c0f-6c7f-4ccf-9632-82a6bb184522 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:01.497162 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873c312-f90f-4db2-8dab-05717ff00008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:01.497869 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873c312-f90f-4db2-8dab-05717ff00008 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:01.498568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873c312-f90f-4db2-8dab-05717ff00008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:01.498568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873c312-f90f-4db2-8dab-05717ff00008 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:01.500379 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873c312-f90f-4db2-8dab-05717ff00008 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:23:01.500379 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873c312-f90f-4db2-8dab-05717ff00008 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:01.501739 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873c312-f90f-4db2-8dab-05717ff00008 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:01.519016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1188/2358] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:23:01.520569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888f049-75e7-42f8-ae16-6251ad4b20c8 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:01.534419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888f049-75e7-42f8-ae16-6251ad4b20c8 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:01.540681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3851c6-7e36-48fe-b9f6-b632f5f0b7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:01.541618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3851c6-7e36-48fe-b9f6-b632f5f0b7bd admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:01.541895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3851c6-7e36-48fe-b9f6-b632f5f0b7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:01.542268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3851c6-7e36-48fe-b9f6-b632f5f0b7bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:01.543069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3851c6-7e36-48fe-b9f6-b632f5f0b7bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:01.545261 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3851c6-7e36-48fe-b9f6-b632f5f0b7bd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:01.583649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1171/2359] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:23:01 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:23:01.588368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873c312-f90f-4db2-8dab-05717ff00008 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:01.604261 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873c312-f90f-4db2-8dab-05717ff00008 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:01.682401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd95d1-76b5-4a27-97ab-08f6fb3873f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:01.683492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd95d1-76b5-4a27-97ab-08f6fb3873f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:01.683962 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd95d1-76b5-4a27-97ab-08f6fb3873f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:01.684409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd95d1-76b5-4a27-97ab-08f6fb3873f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:01.685409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd95d1-76b5-4a27-97ab-08f6fb3873f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:01.687979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd95d1-76b5-4a27-97ab-08f6fb3873f1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:01.722348 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1189/2360] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:23:01 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:23:01.727990 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3851c6-7e36-48fe-b9f6-b632f5f0b7bd admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:01.746120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3851c6-7e36-48fe-b9f6-b632f5f0b7bd admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:01.755522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08e9e2-70cd-46a5-af2a-3807a8f02f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:01.756187 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08e9e2-70cd-46a5-af2a-3807a8f02f79 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:01.756614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08e9e2-70cd-46a5-af2a-3807a8f02f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:01.757093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08e9e2-70cd-46a5-af2a-3807a8f02f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:01.758303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08e9e2-70cd-46a5-af2a-3807a8f02f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:01.761933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08e9e2-70cd-46a5-af2a-3807a8f02f79 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:01.784619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1172/2361] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:01.788921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd95d1-76b5-4a27-97ab-08f6fb3873f1 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:01.807668 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd95d1-76b5-4a27-97ab-08f6fb3873f1 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:01.813253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ffa18-47db-45ce-8eab-141c0bb7335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:01.814160 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ffa18-47db-45ce-8eab-141c0bb7335e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:01.814580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ffa18-47db-45ce-8eab-141c0bb7335e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:01.815190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ffa18-47db-45ce-8eab-141c0bb7335e admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:01.816110 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ffa18-47db-45ce-8eab-141c0bb7335e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:01.817998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ffa18-47db-45ce-8eab-141c0bb7335e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:01.829660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1190/2362] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:01.834022 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08e9e2-70cd-46a5-af2a-3807a8f02f79 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:01.846820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08e9e2-70cd-46a5-af2a-3807a8f02f79 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:01.853194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea21063-95ef-4ac7-96e9-605aa54cbf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:01.854744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea21063-95ef-4ac7-96e9-605aa54cbf46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:01.855259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea21063-95ef-4ac7-96e9-605aa54cbf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:01.855615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea21063-95ef-4ac7-96e9-605aa54cbf46 admin admin] PATH_INFO: `/v3/projects/4f49540028c649c6814a5662d387ab7c/users/ae265b5e580545de9c9ae6514fc1f7c4/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:01.872478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea21063-95ef-4ac7-96e9-605aa54cbf46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f49540028c649c6814a5662d387ab7c, user_id=ae265b5e580545de9c9ae6514fc1f7c4, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:01.875557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea21063-95ef-4ac7-96e9-605aa54cbf46 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:01.903216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1173/2363] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:01 2022] PUT /identity/v3/projects/4f49540028c649c6814a5662d387ab7c/users/ae265b5e580545de9c9ae6514fc1f7c4/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:23:01.914142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989e522-8701-4029-ac74-3eb3b2b1f3f3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:01.915011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989e522-8701-4029-ac74-3eb3b2b1f3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:01.915498 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989e522-8701-4029-ac74-3eb3b2b1f3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:01.960104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5989e522-8701-4029-ac74-3eb3b2b1f3f3 None None] MFA Rules not processed for user `ae265b5e580545de9c9ae6514fc1f7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:01.961573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5989e522-8701-4029-ac74-3eb3b2b1f3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:02.048394 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1191/2364] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:02.057455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea21063-95ef-4ac7-96e9-605aa54cbf46 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:02.059403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cea21063-95ef-4ac7-96e9-605aa54cbf46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:02.195655 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989e522-8701-4029-ac74-3eb3b2b1f3f3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:02.197515 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5989e522-8701-4029-ac74-3eb3b2b1f3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:02.204085 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea21063-95ef-4ac7-96e9-605aa54cbf46 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:02.212325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa136fad-21e2-4bfd-9b98-a0960ad4f447 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:02.213011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa136fad-21e2-4bfd-9b98-a0960ad4f447 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:02.213282 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa136fad-21e2-4bfd-9b98-a0960ad4f447 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:02.213574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa136fad-21e2-4bfd-9b98-a0960ad4f447 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:02.220761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:02.225789 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa136fad-21e2-4bfd-9b98-a0960ad4f447 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:02.229512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa136fad-21e2-4bfd-9b98-a0960ad4f447 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:02.242172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1174/2365] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:02.279544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:02.284335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989e522-8701-4029-ac74-3eb3b2b1f3f3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:02.290226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403156a0-b053-411e-b8ad-a2beaf1b4104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:02.291054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403156a0-b053-411e-b8ad-a2beaf1b4104 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:02.291432 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403156a0-b053-411e-b8ad-a2beaf1b4104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:02.292018 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403156a0-b053-411e-b8ad-a2beaf1b4104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:02.294056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-403156a0-b053-411e-b8ad-a2beaf1b4104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:02.305647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403156a0-b053-411e-b8ad-a2beaf1b4104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:02.306772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403156a0-b053-411e-b8ad-a2beaf1b4104 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:02.320614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1192/2366] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:23:05.596603 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa136fad-21e2-4bfd-9b98-a0960ad4f447 service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:05.601396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-fa136fad-21e2-4bfd-9b98-a0960ad4f447 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:05.679577 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa136fad-21e2-4bfd-9b98-a0960ad4f447 service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:05.684776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:05.685640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:05.685971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:05.686235 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] PATH_INFO: `/v3/users/5462b6a8d72749b9a07cd52ff8cb29d1` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:05.687427 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5462b6a8d72749b9a07cd52ff8cb29d1)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:05.689354 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:05.738520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5462b6a8d72749b9a07cd52ff8cb29d1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:05.746750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5462b6a8d72749b9a07cd52ff8cb29d1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:05.759753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5462b6a8d72749b9a07cd52ff8cb29d1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:05.760664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5462b6a8d72749b9a07cd52ff8cb29d1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:05.767606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5462b6a8d72749b9a07cd52ff8cb29d1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:05.773713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1175/2367] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:23:05 2022] DELETE /identity/v3/users/5462b6a8d72749b9a07cd52ff8cb29d1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:23:05.953978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403156a0-b053-411e-b8ad-a2beaf1b4104 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:05.972334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403156a0-b053-411e-b8ad-a2beaf1b4104 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:06.039551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:06.040848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:06.041302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:06.041790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] PATH_INFO: `/v3/projects/d08521617c884fe9a67609887f0ddaae` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:06.043465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d08521617c884fe9a67609887f0ddaae)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:06.045554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:06.102964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd08521617c884fe9a67609887f0ddaae', 'request_id': 'req-f09e7075-d5fc-4d90-ad2a-53fc28472846'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.110478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd08521617c884fe9a67609887f0ddaae', 'request_id': 'req-f09e7075-d5fc-4d90-ad2a-53fc28472846'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.121456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd08521617c884fe9a67609887f0ddaae', 'request_id': 'req-f09e7075-d5fc-4d90-ad2a-53fc28472846'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.136827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] The token cache is being invalidate because project d08521617c884fe9a67609887f0ddaae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:23:06.137361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.138285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.142961 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1193/2368] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:23:05 2022] DELETE /identity/v3/projects/d08521617c884fe9a67609887f0ddaae => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:06.146681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:06.148226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:06.162601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcb5e6-86b8-474f-a391-021817f77d6d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:06.229220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:06.230229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:06.230590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:06.230869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] PATH_INFO: `/v3/users/747fb356fcab40e2b96926f3a120f022` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:06.232123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=747fb356fcab40e2b96926f3a120f022)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:06.234130 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:06.248863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:06.289604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '747fb356fcab40e2b96926f3a120f022'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.299927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '747fb356fcab40e2b96926f3a120f022'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.313284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '747fb356fcab40e2b96926f3a120f022'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.314047 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '747fb356fcab40e2b96926f3a120f022'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.320988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '747fb356fcab40e2b96926f3a120f022'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.327666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1176/2369] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:23:06 2022] DELETE /identity/v3/users/747fb356fcab40e2b96926f3a120f022 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:06.513608 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:06.515749 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:06.531740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e7075-d5fc-4d90-ad2a-53fc28472846 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:06.591085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e4857-1f83-4235-a83a-319fb58cda5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:06.591878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e4857-1f83-4235-a83a-319fb58cda5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:06.592177 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e4857-1f83-4235-a83a-319fb58cda5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:06.592437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e4857-1f83-4235-a83a-319fb58cda5f admin admin] PATH_INFO: `/v3/projects/6ea0f3f9ce1949beb362a242a84d9400` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:06.593627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e4857-1f83-4235-a83a-319fb58cda5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ea0f3f9ce1949beb362a242a84d9400)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:06.595656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e4857-1f83-4235-a83a-319fb58cda5f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:06.624513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:06.650134 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-977e4857-1f83-4235-a83a-319fb58cda5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ea0f3f9ce1949beb362a242a84d9400', 'request_id': 'req-977e4857-1f83-4235-a83a-319fb58cda5f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.656269 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-977e4857-1f83-4235-a83a-319fb58cda5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ea0f3f9ce1949beb362a242a84d9400', 'request_id': 'req-977e4857-1f83-4235-a83a-319fb58cda5f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.661639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-977e4857-1f83-4235-a83a-319fb58cda5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ea0f3f9ce1949beb362a242a84d9400', 'request_id': 'req-977e4857-1f83-4235-a83a-319fb58cda5f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.674457 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-977e4857-1f83-4235-a83a-319fb58cda5f admin admin] The token cache is being invalidate because project 6ea0f3f9ce1949beb362a242a84d9400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:23:06.674745 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-977e4857-1f83-4235-a83a-319fb58cda5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.675643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-977e4857-1f83-4235-a83a-319fb58cda5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:06.677487 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1194/2370] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:23:06 2022] DELETE /identity/v3/projects/6ea0f3f9ce1949beb362a242a84d9400 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:09.826805 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:09.827236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:09.839969 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7e720-af7c-49b2-a545-16146a2f8c5f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:09.895529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:09.896390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:09.896670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:09.896949 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] PATH_INFO: `/v3/users/ae265b5e580545de9c9ae6514fc1f7c4` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:09.898162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae265b5e580545de9c9ae6514fc1f7c4)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:09.900104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:09.947590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae265b5e580545de9c9ae6514fc1f7c4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:09.954516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae265b5e580545de9c9ae6514fc1f7c4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:09.971628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae265b5e580545de9c9ae6514fc1f7c4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:09.972349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae265b5e580545de9c9ae6514fc1f7c4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:09.980139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae265b5e580545de9c9ae6514fc1f7c4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:09.986243 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1177/2371] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:23:09 2022] DELETE /identity/v3/users/ae265b5e580545de9c9ae6514fc1f7c4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:09.990756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f069c-7a44-4571-80e5-9a3494a40f90 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:09.991210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f069c-7a44-4571-80e5-9a3494a40f90 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:09.991368 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f069c-7a44-4571-80e5-9a3494a40f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:10.010195 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-fe4f069c-7a44-4571-80e5-9a3494a40f90 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:10.011390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-fe4f069c-7a44-4571-80e5-9a3494a40f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:10.069006 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1195/2372] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:10.079661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:10.081061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:10.094766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:10.140585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c1edc-5400-40a9-8cf9-da2b62ab975a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:10.144266 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36fbc4-e9b3-4f14-a405-9fd4f6c191df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:10.144904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36fbc4-e9b3-4f14-a405-9fd4f6c191df service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:10.145184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36fbc4-e9b3-4f14-a405-9fd4f6c191df service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:10.145456 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36fbc4-e9b3-4f14-a405-9fd4f6c191df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:10.154542 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36fbc4-e9b3-4f14-a405-9fd4f6c191df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:10.155997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36fbc4-e9b3-4f14-a405-9fd4f6c191df service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:10.165969 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1178/2373] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:10.435627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f069c-7a44-4571-80e5-9a3494a40f90 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:10.437095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-fe4f069c-7a44-4571-80e5-9a3494a40f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:10.450729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f069c-7a44-4571-80e5-9a3494a40f90 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:10.455360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:10.456220 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:10.456506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:10.456776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6 admin admin] PATH_INFO: `/v3/projects/4f49540028c649c6814a5662d387ab7c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:10.458045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f49540028c649c6814a5662d387ab7c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:10.460055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:10.506197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f49540028c649c6814a5662d387ab7c', 'request_id': 'req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:10.513604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f49540028c649c6814a5662d387ab7c', 'request_id': 'req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:10.519136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f49540028c649c6814a5662d387ab7c', 'request_id': 'req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:10.532262 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6 admin admin] The token cache is being invalidate because project 4f49540028c649c6814a5662d387ab7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:23:10.532552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:10.533185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d2ae1171-82cc-4bd6-9687-5db812bbc8b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:10.534848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1196/2374] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:23:10 2022] DELETE /identity/v3/projects/4f49540028c649c6814a5662d387ab7c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:10.539766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e8e9c-4582-4641-bbf7-52d64e502436 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:10.540237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e8e9c-4582-4641-bbf7-52d64e502436 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:10.540714 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e8e9c-4582-4641-bbf7-52d64e502436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:10.559156 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-666e8e9c-4582-4641-bbf7-52d64e502436 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:10.560356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-666e8e9c-4582-4641-bbf7-52d64e502436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:10.619691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1179/2375] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:10.628399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aeeeb9-22b7-4f1e-87fd-6cf233a09f52 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:10.628689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aeeeb9-22b7-4f1e-87fd-6cf233a09f52 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:10.628968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aeeeb9-22b7-4f1e-87fd-6cf233a09f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:10.647273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a9aeeeb9-22b7-4f1e-87fd-6cf233a09f52 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:10.648484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a9aeeeb9-22b7-4f1e-87fd-6cf233a09f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:10.657187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1197/2376] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:10.661340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e8e9c-4582-4641-bbf7-52d64e502436 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:10.671716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e8e9c-4582-4641-bbf7-52d64e502436 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:10.676554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33956b89-379f-494f-8b66-7a7517b88e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:10.677200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33956b89-379f-494f-8b66-7a7517b88e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:10.677477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33956b89-379f-494f-8b66-7a7517b88e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:10.677772 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33956b89-379f-494f-8b66-7a7517b88e17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:10.678490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33956b89-379f-494f-8b66-7a7517b88e17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:23:10.678790 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33956b89-379f-494f-8b66-7a7517b88e17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:10.680476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33956b89-379f-494f-8b66-7a7517b88e17 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:10.692706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1180/2377] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:23:10.698075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aeeeb9-22b7-4f1e-87fd-6cf233a09f52 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:10.707495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aeeeb9-22b7-4f1e-87fd-6cf233a09f52 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:10.712792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778649d-a7ab-448d-be76-f59f0d33fadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:10.713444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778649d-a7ab-448d-be76-f59f0d33fadb admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:10.713755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778649d-a7ab-448d-be76-f59f0d33fadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:10.714028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778649d-a7ab-448d-be76-f59f0d33fadb admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:10.714716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778649d-a7ab-448d-be76-f59f0d33fadb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:10.716636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778649d-a7ab-448d-be76-f59f0d33fadb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:10.741951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1198/2378] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:23:10 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:23:10.746261 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33956b89-379f-494f-8b66-7a7517b88e17 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:10.756561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33956b89-379f-494f-8b66-7a7517b88e17 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:10.813888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff489b-853d-414e-a55a-90359471fc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:10.814483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff489b-853d-414e-a55a-90359471fc9c admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:10.814818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff489b-853d-414e-a55a-90359471fc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:10.815129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff489b-853d-414e-a55a-90359471fc9c admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:10.815819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff489b-853d-414e-a55a-90359471fc9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:10.817755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff489b-853d-414e-a55a-90359471fc9c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:10.838379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1181/2379] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:23:10 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:23:10.842513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778649d-a7ab-448d-be76-f59f0d33fadb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:10.852919 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778649d-a7ab-448d-be76-f59f0d33fadb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:10.857553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705646a9-1ba6-4b86-9e60-78ed6867b1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:10.858240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705646a9-1ba6-4b86-9e60-78ed6867b1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:10.858523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705646a9-1ba6-4b86-9e60-78ed6867b1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:10.858782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705646a9-1ba6-4b86-9e60-78ed6867b1c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:10.859495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705646a9-1ba6-4b86-9e60-78ed6867b1c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:10.861084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705646a9-1ba6-4b86-9e60-78ed6867b1c3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:10.862340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:10.873879 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1199/2380] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:10.877188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff489b-853d-414e-a55a-90359471fc9c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:10.887800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff489b-853d-414e-a55a-90359471fc9c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:10.892368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1993b-9c17-46b5-9b3b-cf985b99a1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:10.893012 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1993b-9c17-46b5-9b3b-cf985b99a1fd admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:10.893291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1993b-9c17-46b5-9b3b-cf985b99a1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:10.893564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1993b-9c17-46b5-9b3b-cf985b99a1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:10.894306 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1993b-9c17-46b5-9b3b-cf985b99a1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:10.895897 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1993b-9c17-46b5-9b3b-cf985b99a1fd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:10.906590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1182/2381] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:23:10.911033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705646a9-1ba6-4b86-9e60-78ed6867b1c3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:10.921084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705646a9-1ba6-4b86-9e60-78ed6867b1c3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:10.925894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bb3dd-ea34-46f1-8cf1-28081d00c8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:10.926748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bb3dd-ea34-46f1-8cf1-28081d00c8f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:10.927021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bb3dd-ea34-46f1-8cf1-28081d00c8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:10.927286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bb3dd-ea34-46f1-8cf1-28081d00c8f3 admin admin] PATH_INFO: `/v3/projects/97f9489fba044b7ab4a27b197b017b06/users/87efe160534a48cdb253dd24245213e4/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:10.941330 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bb3dd-ea34-46f1-8cf1-28081d00c8f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f9489fba044b7ab4a27b197b017b06, user_id=87efe160534a48cdb253dd24245213e4, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:10.944011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bb3dd-ea34-46f1-8cf1-28081d00c8f3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:10.958400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1200/2382] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:10 2022] PUT /identity/v3/projects/97f9489fba044b7ab4a27b197b017b06/users/87efe160534a48cdb253dd24245213e4/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:10.969528 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6786977-bf8f-4b36-bedd-b3ed975a1ba5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:10.969844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6786977-bf8f-4b36-bedd-b3ed975a1ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:10.970118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6786977-bf8f-4b36-bedd-b3ed975a1ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:10.997229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f6786977-bf8f-4b36-bedd-b3ed975a1ba5 None None] MFA Rules not processed for user `87efe160534a48cdb253dd24245213e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:10.998495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f6786977-bf8f-4b36-bedd-b3ed975a1ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:11.006645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:11.066445 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1183/2383] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:11.079022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bb3dd-ea34-46f1-8cf1-28081d00c8f3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:11.080753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-540bb3dd-ea34-46f1-8cf1-28081d00c8f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:11.159764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bb3dd-ea34-46f1-8cf1-28081d00c8f3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:11.165878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15033c93-5859-4941-adef-2b906b51d3bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:11.166839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15033c93-5859-4941-adef-2b906b51d3bb service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:11.167116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15033c93-5859-4941-adef-2b906b51d3bb service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:11.167387 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15033c93-5859-4941-adef-2b906b51d3bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:11.175212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15033c93-5859-4941-adef-2b906b51d3bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:11.177058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15033c93-5859-4941-adef-2b906b51d3bb service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:11.186474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1201/2384] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:11.391930 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6786977-bf8f-4b36-bedd-b3ed975a1ba5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:11.453904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6786977-bf8f-4b36-bedd-b3ed975a1ba5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:11.458606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:11.459339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:11.459620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:11.459889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] PATH_INFO: `/v3/users/87efe160534a48cdb253dd24245213e4` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:11.461060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87efe160534a48cdb253dd24245213e4)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:11.463201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:11.509672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87efe160534a48cdb253dd24245213e4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:11.517082 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87efe160534a48cdb253dd24245213e4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:11.529471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87efe160534a48cdb253dd24245213e4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:11.530093 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87efe160534a48cdb253dd24245213e4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:11.537046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87efe160534a48cdb253dd24245213e4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:11.548368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1184/2385] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:23:11 2022] DELETE /identity/v3/users/87efe160534a48cdb253dd24245213e4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:11.553110 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed7245-8114-4d6c-8cdf-ed9fafd2a293 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:11.553414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed7245-8114-4d6c-8cdf-ed9fafd2a293 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:11.553671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed7245-8114-4d6c-8cdf-ed9fafd2a293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:11.571890 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-7bed7245-8114-4d6c-8cdf-ed9fafd2a293 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:11.573021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7bed7245-8114-4d6c-8cdf-ed9fafd2a293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:11.631558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1202/2386] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:11.642590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:11.644038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:11.676849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:11.704141 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08e65a-e7f1-4f37-bb7c-e922ef4f3d98 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:11.707748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b27ab8-d08a-496b-a0b3-4418cb74a80c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:11.708386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b27ab8-d08a-496b-a0b3-4418cb74a80c service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:11.708665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b27ab8-d08a-496b-a0b3-4418cb74a80c service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:11.708936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b27ab8-d08a-496b-a0b3-4418cb74a80c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:11.717911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b27ab8-d08a-496b-a0b3-4418cb74a80c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:11.719330 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b27ab8-d08a-496b-a0b3-4418cb74a80c service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:11.729272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1185/2387] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:12.001474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed7245-8114-4d6c-8cdf-ed9fafd2a293 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:12.003121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7bed7245-8114-4d6c-8cdf-ed9fafd2a293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:12.016940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed7245-8114-4d6c-8cdf-ed9fafd2a293 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:12.022152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b184df3-2303-4e7f-985c-3be0afe24a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:12.023023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b184df3-2303-4e7f-985c-3be0afe24a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.023306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b184df3-2303-4e7f-985c-3be0afe24a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.023580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b184df3-2303-4e7f-985c-3be0afe24a54 admin admin] PATH_INFO: `/v3/projects/97f9489fba044b7ab4a27b197b017b06` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.025525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b184df3-2303-4e7f-985c-3be0afe24a54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f9489fba044b7ab4a27b197b017b06)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:12.027907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b184df3-2303-4e7f-985c-3be0afe24a54 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:12.076846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5b184df3-2303-4e7f-985c-3be0afe24a54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f9489fba044b7ab4a27b197b017b06', 'request_id': 'req-5b184df3-2303-4e7f-985c-3be0afe24a54'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:12.078700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:12.083646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5b184df3-2303-4e7f-985c-3be0afe24a54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f9489fba044b7ab4a27b197b017b06', 'request_id': 'req-5b184df3-2303-4e7f-985c-3be0afe24a54'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:12.092040 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5b184df3-2303-4e7f-985c-3be0afe24a54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f9489fba044b7ab4a27b197b017b06', 'request_id': 'req-5b184df3-2303-4e7f-985c-3be0afe24a54'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:12.105298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5b184df3-2303-4e7f-985c-3be0afe24a54 admin admin] The token cache is being invalidate because project 97f9489fba044b7ab4a27b197b017b06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:23:12.105661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5b184df3-2303-4e7f-985c-3be0afe24a54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:12.106239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5b184df3-2303-4e7f-985c-3be0afe24a54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:12.107976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1203/2388] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:23:12 2022] DELETE /identity/v3/projects/97f9489fba044b7ab4a27b197b017b06 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:12.112886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c6fbd-946e-4f63-8714-ab0dcc535284 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.113455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c6fbd-946e-4f63-8714-ab0dcc535284 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.113556 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c6fbd-946e-4f63-8714-ab0dcc535284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.131651 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f35c6fbd-946e-4f63-8714-ab0dcc535284 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:12.132849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f35c6fbd-946e-4f63-8714-ab0dcc535284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:12.192225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1186/2389] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:12.199980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4d2a7-770d-465d-98aa-a5cde51e3d29 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.200265 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4d2a7-770d-465d-98aa-a5cde51e3d29 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.200541 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4d2a7-770d-465d-98aa-a5cde51e3d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.219492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-cfb4d2a7-770d-465d-98aa-a5cde51e3d29 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:12.220691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cfb4d2a7-770d-465d-98aa-a5cde51e3d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:12.229404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1204/2390] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:12.233585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c6fbd-946e-4f63-8714-ab0dcc535284 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:12.243170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c6fbd-946e-4f63-8714-ab0dcc535284 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:12.247717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c26ef-eb15-4b76-801f-5c699f077b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:12.248372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c26ef-eb15-4b76-801f-5c699f077b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.248656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c26ef-eb15-4b76-801f-5c699f077b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.248929 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c26ef-eb15-4b76-801f-5c699f077b7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.249649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c26ef-eb15-4b76-801f-5c699f077b7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:23:12.249983 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c26ef-eb15-4b76-801f-5c699f077b7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:12.251582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c26ef-eb15-4b76-801f-5c699f077b7f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:12.270897 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1187/2391] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:23:12.271835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4d2a7-770d-465d-98aa-a5cde51e3d29 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:12.283538 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4d2a7-770d-465d-98aa-a5cde51e3d29 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:12.288595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee302fe-1922-4e41-90eb-56dad5db9d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:12.289271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee302fe-1922-4e41-90eb-56dad5db9d50 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.289656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee302fe-1922-4e41-90eb-56dad5db9d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.290482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee302fe-1922-4e41-90eb-56dad5db9d50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.291127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee302fe-1922-4e41-90eb-56dad5db9d50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:12.293182 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee302fe-1922-4e41-90eb-56dad5db9d50 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:12.324506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1205/2392] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:23:12 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:23:12.325787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c26ef-eb15-4b76-801f-5c699f077b7f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:12.335933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c26ef-eb15-4b76-801f-5c699f077b7f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:12.393244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216677-56a7-40ec-ab9a-6d7f3c768481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:12.393973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216677-56a7-40ec-ab9a-6d7f3c768481 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.394287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216677-56a7-40ec-ab9a-6d7f3c768481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.394497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216677-56a7-40ec-ab9a-6d7f3c768481 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.395170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e216677-56a7-40ec-ab9a-6d7f3c768481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:12.397149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e216677-56a7-40ec-ab9a-6d7f3c768481 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:12.420646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1188/2393] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:23:12 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:23:12.424904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee302fe-1922-4e41-90eb-56dad5db9d50 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:12.435752 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee302fe-1922-4e41-90eb-56dad5db9d50 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:12.440884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a31902-63e1-4300-9b38-c3b90cdeabbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:12.441557 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a31902-63e1-4300-9b38-c3b90cdeabbc admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.441913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a31902-63e1-4300-9b38-c3b90cdeabbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.442216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a31902-63e1-4300-9b38-c3b90cdeabbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.443001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a31902-63e1-4300-9b38-c3b90cdeabbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:12.444726 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a31902-63e1-4300-9b38-c3b90cdeabbc admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:12.460183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1206/2394] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:12.462072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216677-56a7-40ec-ab9a-6d7f3c768481 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:12.473193 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216677-56a7-40ec-ab9a-6d7f3c768481 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:12.484066 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a98fa-9536-4663-a942-f7d2eb8cebcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:12.484902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a98fa-9536-4663-a942-f7d2eb8cebcc admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.485188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a98fa-9536-4663-a942-f7d2eb8cebcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.485496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a98fa-9536-4663-a942-f7d2eb8cebcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.486315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a98fa-9536-4663-a942-f7d2eb8cebcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:12.488205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a98fa-9536-4663-a942-f7d2eb8cebcc admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:12.504031 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1189/2395] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:12.505040 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a31902-63e1-4300-9b38-c3b90cdeabbc admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:12.516629 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a31902-63e1-4300-9b38-c3b90cdeabbc admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:12.525202 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec60ea-127b-4e03-9926-b0cc5cd45103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:12.526211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec60ea-127b-4e03-9926-b0cc5cd45103 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.526557 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec60ea-127b-4e03-9926-b0cc5cd45103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.526909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec60ea-127b-4e03-9926-b0cc5cd45103 admin admin] PATH_INFO: `/v3/projects/432b9694b1ac4778bcf8f93b677d158b/users/b9f5fe0bff2147b09505f2a634cc6526/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.545124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec60ea-127b-4e03-9926-b0cc5cd45103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=432b9694b1ac4778bcf8f93b677d158b, user_id=b9f5fe0bff2147b09505f2a634cc6526, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:12.547797 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec60ea-127b-4e03-9926-b0cc5cd45103 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:12.562919 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1207/2396] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:12 2022] PUT /identity/v3/projects/432b9694b1ac4778bcf8f93b677d158b/users/b9f5fe0bff2147b09505f2a634cc6526/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:23:12.574660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c93773-88b9-4639-9453-16a194555312 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.574934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c93773-88b9-4639-9453-16a194555312 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.575209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c93773-88b9-4639-9453-16a194555312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.605486 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-87c93773-88b9-4639-9453-16a194555312 None None] MFA Rules not processed for user `b9f5fe0bff2147b09505f2a634cc6526`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:12.606732 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-87c93773-88b9-4639-9453-16a194555312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:12.635182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:12.676746 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1190/2397] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:12.682108 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec60ea-127b-4e03-9926-b0cc5cd45103 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:12.741950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec60ea-127b-4e03-9926-b0cc5cd45103 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:12.747413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5cebb-25e8-4436-a32c-e7a2fb5e1b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:12.748075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5cebb-25e8-4436-a32c-e7a2fb5e1b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.748358 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5cebb-25e8-4436-a32c-e7a2fb5e1b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.748640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5cebb-25e8-4436-a32c-e7a2fb5e1b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.749325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5cebb-25e8-4436-a32c-e7a2fb5e1b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:12.751305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5cebb-25e8-4436-a32c-e7a2fb5e1b2a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:12.777924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1208/2398] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:23:12 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:23:12.781850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c93773-88b9-4639-9453-16a194555312 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:12.793441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c93773-88b9-4639-9453-16a194555312 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:12.850160 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc8f8-bd20-484a-b8b2-f135077f084f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:12.850758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc8f8-bd20-484a-b8b2-f135077f084f admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.851043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc8f8-bd20-484a-b8b2-f135077f084f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.851319 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc8f8-bd20-484a-b8b2-f135077f084f admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.852056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcc8f8-bd20-484a-b8b2-f135077f084f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:12.853989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcc8f8-bd20-484a-b8b2-f135077f084f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:12.873737 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1191/2399] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:23:12 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:23:12.877742 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5cebb-25e8-4436-a32c-e7a2fb5e1b2a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:12.888772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5cebb-25e8-4436-a32c-e7a2fb5e1b2a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:12.893360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2643d3-e776-4fb9-9611-a79a6812f661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:12.894043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2643d3-e776-4fb9-9611-a79a6812f661 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.894327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2643d3-e776-4fb9-9611-a79a6812f661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.894603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2643d3-e776-4fb9-9611-a79a6812f661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.895318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2643d3-e776-4fb9-9611-a79a6812f661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:12.896940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2643d3-e776-4fb9-9611-a79a6812f661 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:12.907571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1209/2400] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:23:12.911649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc8f8-bd20-484a-b8b2-f135077f084f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:12.921754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc8f8-bd20-484a-b8b2-f135077f084f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:12.926607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f51e31-9ef0-4ff0-bc03-24f701a2526c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:12.927474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f51e31-9ef0-4ff0-bc03-24f701a2526c admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:12.927755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f51e31-9ef0-4ff0-bc03-24f701a2526c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:12.928028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f51e31-9ef0-4ff0-bc03-24f701a2526c admin admin] PATH_INFO: `/v3/projects/1ce87d75df8d449dbed22061d749ea7a/users/c45492bab17d42c0874f91e3e85709e7/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:12.943166 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f51e31-9ef0-4ff0-bc03-24f701a2526c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce87d75df8d449dbed22061d749ea7a, user_id=c45492bab17d42c0874f91e3e85709e7, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:12.946451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f51e31-9ef0-4ff0-bc03-24f701a2526c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:12.958938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1192/2401] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:12 2022] PUT /identity/v3/projects/1ce87d75df8d449dbed22061d749ea7a/users/c45492bab17d42c0874f91e3e85709e7/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:23:12.962969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2643d3-e776-4fb9-9611-a79a6812f661 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:12.972715 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2643d3-e776-4fb9-9611-a79a6812f661 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:13.019641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:13.031300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0eed6-eb50-490a-bbcf-0c0dd2bca9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:13.031968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0eed6-eb50-490a-bbcf-0c0dd2bca9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:13.032251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0eed6-eb50-490a-bbcf-0c0dd2bca9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:13.032523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0eed6-eb50-490a-bbcf-0c0dd2bca9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:13.033238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0eed6-eb50-490a-bbcf-0c0dd2bca9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:13.034915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0eed6-eb50-490a-bbcf-0c0dd2bca9ee admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:13.044760 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1210/2402] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:13.049075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f51e31-9ef0-4ff0-bc03-24f701a2526c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:13.060154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f51e31-9ef0-4ff0-bc03-24f701a2526c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:13.064767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638e682-98ef-468f-ac52-7dfc07c615c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:13.065636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638e682-98ef-468f-ac52-7dfc07c615c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:13.065954 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638e682-98ef-468f-ac52-7dfc07c615c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:13.066247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638e682-98ef-468f-ac52-7dfc07c615c8 admin admin] PATH_INFO: `/v3/projects/1ce87d75df8d449dbed22061d749ea7a/users/c45492bab17d42c0874f91e3e85709e7/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:13.068195 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638e682-98ef-468f-ac52-7dfc07c615c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce87d75df8d449dbed22061d749ea7a, user_id=c45492bab17d42c0874f91e3e85709e7, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:13.071562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638e682-98ef-468f-ac52-7dfc07c615c8 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:13.081700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1193/2403] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:13 2022] PUT /identity/v3/projects/1ce87d75df8d449dbed22061d749ea7a/users/c45492bab17d42c0874f91e3e85709e7/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:13.092099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e5edf-1cf5-4d30-8f4b-640d066fcc66 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:13.092579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e5edf-1cf5-4d30-8f4b-640d066fcc66 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:13.092681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e5edf-1cf5-4d30-8f4b-640d066fcc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:13.121124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-625e5edf-1cf5-4d30-8f4b-640d066fcc66 None None] MFA Rules not processed for user `c45492bab17d42c0874f91e3e85709e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:13.122300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-625e5edf-1cf5-4d30-8f4b-640d066fcc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:13.188746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1211/2404] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:13 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:13.200313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638e682-98ef-468f-ac52-7dfc07c615c8 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:13.202070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2638e682-98ef-468f-ac52-7dfc07c615c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:13.259132 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638e682-98ef-468f-ac52-7dfc07c615c8 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:13.263733 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803963a0-f6a9-42ae-84a1-0d206fd45781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:13.264378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803963a0-f6a9-42ae-84a1-0d206fd45781 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:13.264658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803963a0-f6a9-42ae-84a1-0d206fd45781 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:13.264928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803963a0-f6a9-42ae-84a1-0d206fd45781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:13.275299 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803963a0-f6a9-42ae-84a1-0d206fd45781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:13.276733 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803963a0-f6a9-42ae-84a1-0d206fd45781 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:13.286328 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1194/2405] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:13 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:14.625623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e5edf-1cf5-4d30-8f4b-640d066fcc66 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:14.689124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e5edf-1cf5-4d30-8f4b-640d066fcc66 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:14.693679 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:14.694585 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:14.694861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:14.695129 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] PATH_INFO: `/v3/users/b9f5fe0bff2147b09505f2a634cc6526` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:14.696333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f5fe0bff2147b09505f2a634cc6526)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:14.698462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:14.746770 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f5fe0bff2147b09505f2a634cc6526'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:14.755202 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f5fe0bff2147b09505f2a634cc6526'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:14.755831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f5fe0bff2147b09505f2a634cc6526'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:14.756444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f5fe0bff2147b09505f2a634cc6526'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:14.769629 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f5fe0bff2147b09505f2a634cc6526'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:14.771993 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:14.787298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1212/2406] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:23:14 2022] DELETE /identity/v3/users/b9f5fe0bff2147b09505f2a634cc6526 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:14.789289 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1ed7b-3b23-4a0d-8657-ce02ef7307f9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:14.789675 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1ed7b-3b23-4a0d-8657-ce02ef7307f9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:14.789888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1ed7b-3b23-4a0d-8657-ce02ef7307f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:14.810873 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-58d1ed7b-3b23-4a0d-8657-ce02ef7307f9 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:14.812068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-58d1ed7b-3b23-4a0d-8657-ce02ef7307f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:14.828270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:14.871368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1195/2407] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:23:14.881059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:14.882518 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:14.944206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a793655-e038-49c4-a1d7-b3c4eff894f4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:14.948669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57323c-c27d-440a-b3e8-e87c8b72e012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:14.949326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57323c-c27d-440a-b3e8-e87c8b72e012 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:14.949610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57323c-c27d-440a-b3e8-e87c8b72e012 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:14.949936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57323c-c27d-440a-b3e8-e87c8b72e012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:14.959731 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57323c-c27d-440a-b3e8-e87c8b72e012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:14.961209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57323c-c27d-440a-b3e8-e87c8b72e012 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:14.973453 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1213/2408] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:15.265192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1ed7b-3b23-4a0d-8657-ce02ef7307f9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:15.266712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-58d1ed7b-3b23-4a0d-8657-ce02ef7307f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:15.281199 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1ed7b-3b23-4a0d-8657-ce02ef7307f9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:15.285856 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:15.286799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:15.287564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:15.287846 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] PATH_INFO: `/v3/projects/432b9694b1ac4778bcf8f93b677d158b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:15.289204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=432b9694b1ac4778bcf8f93b677d158b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:15.291708 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:15.341928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '432b9694b1ac4778bcf8f93b677d158b', 'request_id': 'req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.350584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '432b9694b1ac4778bcf8f93b677d158b', 'request_id': 'req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.357356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '432b9694b1ac4778bcf8f93b677d158b', 'request_id': 'req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.367730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] The token cache is being invalidate because project 432b9694b1ac4778bcf8f93b677d158b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:23:15.368015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.368646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.370355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1196/2409] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:23:15 2022] DELETE /identity/v3/projects/432b9694b1ac4778bcf8f93b677d158b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:23:15.374345 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57323c-c27d-440a-b3e8-e87c8b72e012 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:15.383921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57323c-c27d-440a-b3e8-e87c8b72e012 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:15.441234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f2233-d7dd-4f81-be19-7981e9038343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:15.442132 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f2233-d7dd-4f81-be19-7981e9038343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:15.442413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f2233-d7dd-4f81-be19-7981e9038343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:15.442682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f2233-d7dd-4f81-be19-7981e9038343 admin admin] PATH_INFO: `/v3/users/c45492bab17d42c0874f91e3e85709e7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:15.443875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f2233-d7dd-4f81-be19-7981e9038343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c45492bab17d42c0874f91e3e85709e7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:15.445861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f2233-d7dd-4f81-be19-7981e9038343 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:15.491251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0e6f2233-d7dd-4f81-be19-7981e9038343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c45492bab17d42c0874f91e3e85709e7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.498537 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0e6f2233-d7dd-4f81-be19-7981e9038343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c45492bab17d42c0874f91e3e85709e7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.499152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0e6f2233-d7dd-4f81-be19-7981e9038343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c45492bab17d42c0874f91e3e85709e7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.499800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0e6f2233-d7dd-4f81-be19-7981e9038343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c45492bab17d42c0874f91e3e85709e7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.512581 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0e6f2233-d7dd-4f81-be19-7981e9038343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c45492bab17d42c0874f91e3e85709e7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.525033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1214/2410] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:23:15 2022] DELETE /identity/v3/users/c45492bab17d42c0874f91e3e85709e7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:15.796583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:15.798136 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:15.812471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9733db-d9a3-4087-ac5c-ad823beaf1ca admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:15.873079 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0549253-f8dd-4844-9024-81bd4da9ee77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:15.873947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0549253-f8dd-4844-9024-81bd4da9ee77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:15.874226 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0549253-f8dd-4844-9024-81bd4da9ee77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:15.874494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0549253-f8dd-4844-9024-81bd4da9ee77 admin admin] PATH_INFO: `/v3/projects/1ce87d75df8d449dbed22061d749ea7a` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:15.875647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0549253-f8dd-4844-9024-81bd4da9ee77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce87d75df8d449dbed22061d749ea7a)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:15.877524 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0549253-f8dd-4844-9024-81bd4da9ee77 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:15.921284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:15.922557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0549253-f8dd-4844-9024-81bd4da9ee77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce87d75df8d449dbed22061d749ea7a', 'request_id': 'req-c0549253-f8dd-4844-9024-81bd4da9ee77'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.930826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0549253-f8dd-4844-9024-81bd4da9ee77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce87d75df8d449dbed22061d749ea7a', 'request_id': 'req-c0549253-f8dd-4844-9024-81bd4da9ee77'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.938272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0549253-f8dd-4844-9024-81bd4da9ee77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce87d75df8d449dbed22061d749ea7a', 'request_id': 'req-c0549253-f8dd-4844-9024-81bd4da9ee77'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.949248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0549253-f8dd-4844-9024-81bd4da9ee77 admin admin] The token cache is being invalidate because project 1ce87d75df8d449dbed22061d749ea7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:23:15.949530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0549253-f8dd-4844-9024-81bd4da9ee77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.950346 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0549253-f8dd-4844-9024-81bd4da9ee77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:15.951886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1197/2411] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:23:15 2022] DELETE /identity/v3/projects/1ce87d75df8d449dbed22061d749ea7a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:15.956842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3c414-0942-46fd-98c6-b67460b209d4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:15.957291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3c414-0942-46fd-98c6-b67460b209d4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:15.957399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3c414-0942-46fd-98c6-b67460b209d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:15.975543 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f5a3c414-0942-46fd-98c6-b67460b209d4 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:15.976737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f5a3c414-0942-46fd-98c6-b67460b209d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:15.985174 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:16.038940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1215/2412] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:16.048287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5b4e3-3aac-4db2-ba07-8652f3469194 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.048576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5b4e3-3aac-4db2-ba07-8652f3469194 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.048854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5b4e3-3aac-4db2-ba07-8652f3469194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.068144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-55b5b4e3-3aac-4db2-ba07-8652f3469194 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:16.069340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-55b5b4e3-3aac-4db2-ba07-8652f3469194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:16.078368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1198/2413] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:16.082673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3c414-0942-46fd-98c6-b67460b209d4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:16.092583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3c414-0942-46fd-98c6-b67460b209d4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:16.097096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5531d56-7303-4f66-b557-05989fb8c199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:16.097763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5531d56-7303-4f66-b557-05989fb8c199 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.098061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5531d56-7303-4f66-b557-05989fb8c199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.098309 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5531d56-7303-4f66-b557-05989fb8c199 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.099020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5531d56-7303-4f66-b557-05989fb8c199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:23:16.099323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5531d56-7303-4f66-b557-05989fb8c199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:16.100912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5531d56-7303-4f66-b557-05989fb8c199 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:16.113028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1216/2414] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:23:16.117269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5b4e3-3aac-4db2-ba07-8652f3469194 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:16.127167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5b4e3-3aac-4db2-ba07-8652f3469194 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:16.131903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883d34f-22eb-42ef-9a78-3be17bcbf320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:16.132660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883d34f-22eb-42ef-9a78-3be17bcbf320 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.132948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883d34f-22eb-42ef-9a78-3be17bcbf320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.133222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883d34f-22eb-42ef-9a78-3be17bcbf320 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.133947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883d34f-22eb-42ef-9a78-3be17bcbf320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:16.135961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883d34f-22eb-42ef-9a78-3be17bcbf320 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:16.160773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1199/2415] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:23:16 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:23:16.164896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5531d56-7303-4f66-b557-05989fb8c199 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:16.175821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5531d56-7303-4f66-b557-05989fb8c199 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:16.237572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0634204-2bce-4797-93a9-26adfcb27411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:16.238499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0634204-2bce-4797-93a9-26adfcb27411 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.238783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0634204-2bce-4797-93a9-26adfcb27411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.239221 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0634204-2bce-4797-93a9-26adfcb27411 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.239959 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0634204-2bce-4797-93a9-26adfcb27411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:16.241928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0634204-2bce-4797-93a9-26adfcb27411 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:16.263965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1217/2416] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:23:16 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:23:16.268068 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883d34f-22eb-42ef-9a78-3be17bcbf320 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:16.280803 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883d34f-22eb-42ef-9a78-3be17bcbf320 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:16.285487 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31e2ba-d7b8-4879-83fa-2bdb33a7871a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:16.286259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31e2ba-d7b8-4879-83fa-2bdb33a7871a admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.286608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31e2ba-d7b8-4879-83fa-2bdb33a7871a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.286885 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31e2ba-d7b8-4879-83fa-2bdb33a7871a admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.287645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31e2ba-d7b8-4879-83fa-2bdb33a7871a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:16.289240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31e2ba-d7b8-4879-83fa-2bdb33a7871a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:16.300895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1200/2417] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:16.305036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0634204-2bce-4797-93a9-26adfcb27411 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:16.316041 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0634204-2bce-4797-93a9-26adfcb27411 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:16.323200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92912f19-8a9d-4bb7-9805-c1a744be0ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:16.323869 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92912f19-8a9d-4bb7-9805-c1a744be0ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.324151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92912f19-8a9d-4bb7-9805-c1a744be0ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.324422 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92912f19-8a9d-4bb7-9805-c1a744be0ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.325154 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92912f19-8a9d-4bb7-9805-c1a744be0ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:16.326817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92912f19-8a9d-4bb7-9805-c1a744be0ee8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:16.341222 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1218/2418] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:16.345363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31e2ba-d7b8-4879-83fa-2bdb33a7871a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:16.356371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31e2ba-d7b8-4879-83fa-2bdb33a7871a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:16.360991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace194fb-0e67-44c5-a525-e5cc34d66a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:16.361889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace194fb-0e67-44c5-a525-e5cc34d66a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.362168 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace194fb-0e67-44c5-a525-e5cc34d66a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.362437 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace194fb-0e67-44c5-a525-e5cc34d66a5b admin admin] PATH_INFO: `/v3/projects/101b5a2a15534bf9bec56e069dcbdb17/users/dcb2bce848f24d6684b201b862ef1fd1/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.378014 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace194fb-0e67-44c5-a525-e5cc34d66a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=101b5a2a15534bf9bec56e069dcbdb17, user_id=dcb2bce848f24d6684b201b862ef1fd1, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:16.380620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace194fb-0e67-44c5-a525-e5cc34d66a5b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:16.393972 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1201/2419] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:16 2022] PUT /identity/v3/projects/101b5a2a15534bf9bec56e069dcbdb17/users/dcb2bce848f24d6684b201b862ef1fd1/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:16.405038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee9577-c75b-4573-aeda-6deb063c76fd None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.405370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee9577-c75b-4573-aeda-6deb063c76fd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.405616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee9577-c75b-4573-aeda-6deb063c76fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.432580 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f7ee9577-c75b-4573-aeda-6deb063c76fd None None] MFA Rules not processed for user `dcb2bce848f24d6684b201b862ef1fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:16.433814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f7ee9577-c75b-4573-aeda-6deb063c76fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:16.496153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1219/2420] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:16.500741 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace194fb-0e67-44c5-a525-e5cc34d66a5b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:16.563236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace194fb-0e67-44c5-a525-e5cc34d66a5b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:16.568111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d621f9-c52b-4922-987a-269b6496eaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:16.568758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d621f9-c52b-4922-987a-269b6496eaed admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.569034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d621f9-c52b-4922-987a-269b6496eaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.569304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d621f9-c52b-4922-987a-269b6496eaed admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.570030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d621f9-c52b-4922-987a-269b6496eaed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:16.572522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d621f9-c52b-4922-987a-269b6496eaed admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:16.597131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1202/2421] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:23:16 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:23:16.601264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee9577-c75b-4573-aeda-6deb063c76fd None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:16.611921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee9577-c75b-4573-aeda-6deb063c76fd None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:16.616618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:16.669415 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd8d86-8a2e-48c7-9e48-643d184283bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:16.670139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd8d86-8a2e-48c7-9e48-643d184283bb admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.670419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd8d86-8a2e-48c7-9e48-643d184283bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.670719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd8d86-8a2e-48c7-9e48-643d184283bb admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.671411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd8d86-8a2e-48c7-9e48-643d184283bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:16.673370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd8d86-8a2e-48c7-9e48-643d184283bb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:16.693620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1220/2422] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:23:16 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:23:16.698481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d621f9-c52b-4922-987a-269b6496eaed admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:16.708556 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d621f9-c52b-4922-987a-269b6496eaed admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:16.713288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d7090-ed2e-46e2-a154-397ab7f2562b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:16.713986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d7090-ed2e-46e2-a154-397ab7f2562b admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.714266 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d7090-ed2e-46e2-a154-397ab7f2562b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.714538 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d7090-ed2e-46e2-a154-397ab7f2562b admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.715284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d7090-ed2e-46e2-a154-397ab7f2562b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:16.716897 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d7090-ed2e-46e2-a154-397ab7f2562b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:16.727368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1203/2423] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:16.731491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd8d86-8a2e-48c7-9e48-643d184283bb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:16.741189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd8d86-8a2e-48c7-9e48-643d184283bb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:16.746380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecb195-95cc-41fb-82ce-3edba0644e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:16.747241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecb195-95cc-41fb-82ce-3edba0644e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.747520 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecb195-95cc-41fb-82ce-3edba0644e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.747792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecb195-95cc-41fb-82ce-3edba0644e0f admin admin] PATH_INFO: `/v3/projects/029e654d0b9840449f63803e349f0ac0/users/ab8b508e83364d20a9ab3bd487bc7640/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.762999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ecb195-95cc-41fb-82ce-3edba0644e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=029e654d0b9840449f63803e349f0ac0, user_id=ab8b508e83364d20a9ab3bd487bc7640, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:16.765676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ecb195-95cc-41fb-82ce-3edba0644e0f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:16.778586 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1221/2424] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:16 2022] PUT /identity/v3/projects/029e654d0b9840449f63803e349f0ac0/users/ab8b508e83364d20a9ab3bd487bc7640/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:16.782421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d7090-ed2e-46e2-a154-397ab7f2562b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:16.792666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d7090-ed2e-46e2-a154-397ab7f2562b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:16.836725 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:16.850748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa396188-70a8-40fe-8e9d-eaeb21b7891e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:16.851434 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa396188-70a8-40fe-8e9d-eaeb21b7891e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.851710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa396188-70a8-40fe-8e9d-eaeb21b7891e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.851979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa396188-70a8-40fe-8e9d-eaeb21b7891e admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.852709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa396188-70a8-40fe-8e9d-eaeb21b7891e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:16.855160 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa396188-70a8-40fe-8e9d-eaeb21b7891e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:16.864829 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1204/2425] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:16.869554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecb195-95cc-41fb-82ce-3edba0644e0f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:16.879839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecb195-95cc-41fb-82ce-3edba0644e0f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:16.884455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fad2c-c1ab-481e-84f7-72745ea05b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:16.885517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fad2c-c1ab-481e-84f7-72745ea05b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.885827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fad2c-c1ab-481e-84f7-72745ea05b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.886121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fad2c-c1ab-481e-84f7-72745ea05b41 admin admin] PATH_INFO: `/v3/projects/029e654d0b9840449f63803e349f0ac0/users/ab8b508e83364d20a9ab3bd487bc7640/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.888106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fad2c-c1ab-481e-84f7-72745ea05b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=029e654d0b9840449f63803e349f0ac0, user_id=ab8b508e83364d20a9ab3bd487bc7640, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:16.890817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fad2c-c1ab-481e-84f7-72745ea05b41 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:16.901022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1222/2426] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:16 2022] PUT /identity/v3/projects/029e654d0b9840449f63803e349f0ac0/users/ab8b508e83364d20a9ab3bd487bc7640/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:23:16.910914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d69163-58a2-413f-9f31-e9e157bead58 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:16.911204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d69163-58a2-413f-9f31-e9e157bead58 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:16.911478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d69163-58a2-413f-9f31-e9e157bead58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:16.939191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-10d69163-58a2-413f-9f31-e9e157bead58 None None] MFA Rules not processed for user `ab8b508e83364d20a9ab3bd487bc7640`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:16.940396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-10d69163-58a2-413f-9f31-e9e157bead58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:17.011122 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1205/2427] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:17.021909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fad2c-c1ab-481e-84f7-72745ea05b41 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:17.023674 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3f6fad2c-c1ab-481e-84f7-72745ea05b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:17.079667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fad2c-c1ab-481e-84f7-72745ea05b41 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:17.083219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053536e-9d2e-4bb1-9024-dc46b8160702 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:17.083845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053536e-9d2e-4bb1-9024-dc46b8160702 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:17.084123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053536e-9d2e-4bb1-9024-dc46b8160702 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:17.084393 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053536e-9d2e-4bb1-9024-dc46b8160702 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:17.093327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053536e-9d2e-4bb1-9024-dc46b8160702 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:17.094725 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053536e-9d2e-4bb1-9024-dc46b8160702 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:17.104881 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1223/2428] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:17.204954 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d69163-58a2-413f-9f31-e9e157bead58 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:17.215078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d69163-58a2-413f-9f31-e9e157bead58 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:17.218671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966b5d9-8686-4e57-97bb-6ba8a6f54aff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:17.219303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966b5d9-8686-4e57-97bb-6ba8a6f54aff service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:17.219581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966b5d9-8686-4e57-97bb-6ba8a6f54aff service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:17.219855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966b5d9-8686-4e57-97bb-6ba8a6f54aff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:17.276389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966b5d9-8686-4e57-97bb-6ba8a6f54aff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:17.278007 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966b5d9-8686-4e57-97bb-6ba8a6f54aff service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:17.287152 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1206/2429] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:19.871254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053536e-9d2e-4bb1-9024-dc46b8160702 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:19.935743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053536e-9d2e-4bb1-9024-dc46b8160702 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:19.940188 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:19.941039 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:19.941335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:19.941604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] PATH_INFO: `/v3/users/dcb2bce848f24d6684b201b862ef1fd1` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:19.942789 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcb2bce848f24d6684b201b862ef1fd1)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:19.944706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:19.949505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:19.991847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcb2bce848f24d6684b201b862ef1fd1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.000069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcb2bce848f24d6684b201b862ef1fd1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.000704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcb2bce848f24d6684b201b862ef1fd1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.001353 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcb2bce848f24d6684b201b862ef1fd1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.013464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcb2bce848f24d6684b201b862ef1fd1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.025395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1224/2430] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:23:19 2022] DELETE /identity/v3/users/dcb2bce848f24d6684b201b862ef1fd1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:20.030231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83db3c-21ec-4b36-8aad-39ca096e1b5c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:20.030710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83db3c-21ec-4b36-8aad-39ca096e1b5c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:20.030809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83db3c-21ec-4b36-8aad-39ca096e1b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:20.048170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5a83db3c-21ec-4b36-8aad-39ca096e1b5c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:20.049361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5a83db3c-21ec-4b36-8aad-39ca096e1b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:20.108429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1207/2431] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:20.120301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:20.121925 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:20.187945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1f696-4765-4788-9655-4c95560bb33f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:20.191571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77314c74-8009-4f2c-a78f-c6f4c791a09d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:20.192205 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77314c74-8009-4f2c-a78f-c6f4c791a09d service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:20.192480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77314c74-8009-4f2c-a78f-c6f4c791a09d service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:20.192746 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77314c74-8009-4f2c-a78f-c6f4c791a09d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:20.202335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77314c74-8009-4f2c-a78f-c6f4c791a09d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:20.203817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77314c74-8009-4f2c-a78f-c6f4c791a09d service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:20.214463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1225/2432] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:20.480598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83db3c-21ec-4b36-8aad-39ca096e1b5c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:20.482117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5a83db3c-21ec-4b36-8aad-39ca096e1b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:20.495421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83db3c-21ec-4b36-8aad-39ca096e1b5c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:20.500518 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:20.501374 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:20.501647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:20.501945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] PATH_INFO: `/v3/projects/101b5a2a15534bf9bec56e069dcbdb17` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:20.503269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] RBAC: Authorizing `identity:delete_project(project_id=101b5a2a15534bf9bec56e069dcbdb17)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:20.505203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:20.506824 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:20.554950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '101b5a2a15534bf9bec56e069dcbdb17', 'request_id': 'req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.563216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '101b5a2a15534bf9bec56e069dcbdb17', 'request_id': 'req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.569393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '101b5a2a15534bf9bec56e069dcbdb17', 'request_id': 'req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.579746 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] The token cache is being invalidate because project 101b5a2a15534bf9bec56e069dcbdb17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:23:20.580032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.580662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.582245 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1208/2433] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:23:20 2022] DELETE /identity/v3/projects/101b5a2a15534bf9bec56e069dcbdb17 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:23:20.586435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77314c74-8009-4f2c-a78f-c6f4c791a09d service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:20.596646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77314c74-8009-4f2c-a78f-c6f4c791a09d service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:20.653522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ea161-cac8-4739-953e-5e4b3b8cb41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:20.654447 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ea161-cac8-4739-953e-5e4b3b8cb41e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:20.654730 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ea161-cac8-4739-953e-5e4b3b8cb41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:20.655010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ea161-cac8-4739-953e-5e4b3b8cb41e admin admin] PATH_INFO: `/v3/users/ab8b508e83364d20a9ab3bd487bc7640` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:20.656193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ea161-cac8-4739-953e-5e4b3b8cb41e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab8b508e83364d20a9ab3bd487bc7640)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:20.658168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ea161-cac8-4739-953e-5e4b3b8cb41e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:20.688350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:20.707022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d27ea161-cac8-4739-953e-5e4b3b8cb41e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab8b508e83364d20a9ab3bd487bc7640'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.715248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d27ea161-cac8-4739-953e-5e4b3b8cb41e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab8b508e83364d20a9ab3bd487bc7640'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.715960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d27ea161-cac8-4739-953e-5e4b3b8cb41e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab8b508e83364d20a9ab3bd487bc7640'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.716550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d27ea161-cac8-4739-953e-5e4b3b8cb41e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab8b508e83364d20a9ab3bd487bc7640'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.729177 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d27ea161-cac8-4739-953e-5e4b3b8cb41e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab8b508e83364d20a9ab3bd487bc7640'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:20.741517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1226/2434] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:23:20 2022] DELETE /identity/v3/users/ab8b508e83364d20a9ab3bd487bc7640 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:21.102100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:21.104569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:21.120716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590c3d4-31bf-4313-bdf9-b8a33e7c599b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:21.178457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204b5c7-8e4a-42be-924c-fc455e1ccf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:21.179307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204b5c7-8e4a-42be-924c-fc455e1ccf19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:21.179588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204b5c7-8e4a-42be-924c-fc455e1ccf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:21.179868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204b5c7-8e4a-42be-924c-fc455e1ccf19 admin admin] PATH_INFO: `/v3/projects/029e654d0b9840449f63803e349f0ac0` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:21.181038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204b5c7-8e4a-42be-924c-fc455e1ccf19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=029e654d0b9840449f63803e349f0ac0)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:21.183143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204b5c7-8e4a-42be-924c-fc455e1ccf19 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:21.228829 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6204b5c7-8e4a-42be-924c-fc455e1ccf19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '029e654d0b9840449f63803e349f0ac0', 'request_id': 'req-6204b5c7-8e4a-42be-924c-fc455e1ccf19'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:21.236478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6204b5c7-8e4a-42be-924c-fc455e1ccf19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '029e654d0b9840449f63803e349f0ac0', 'request_id': 'req-6204b5c7-8e4a-42be-924c-fc455e1ccf19'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:21.242842 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6204b5c7-8e4a-42be-924c-fc455e1ccf19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '029e654d0b9840449f63803e349f0ac0', 'request_id': 'req-6204b5c7-8e4a-42be-924c-fc455e1ccf19'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:21.253398 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6204b5c7-8e4a-42be-924c-fc455e1ccf19 admin admin] The token cache is being invalidate because project 029e654d0b9840449f63803e349f0ac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:23:21.253682 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6204b5c7-8e4a-42be-924c-fc455e1ccf19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:21.254329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6204b5c7-8e4a-42be-924c-fc455e1ccf19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:21.255871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1209/2435] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:23:21 2022] DELETE /identity/v3/projects/029e654d0b9840449f63803e349f0ac0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:21.260916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc7213-6318-482d-a7f1-cd6e22a0b103 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:21.261395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc7213-6318-482d-a7f1-cd6e22a0b103 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:21.261489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc7213-6318-482d-a7f1-cd6e22a0b103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:21.281162 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3edc7213-6318-482d-a7f1-cd6e22a0b103 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:21.282417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3edc7213-6318-482d-a7f1-cd6e22a0b103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:21.342162 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1227/2436] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:21.350768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f62d4-7256-4aca-b188-d4d15d97aa5a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:21.351053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f62d4-7256-4aca-b188-d4d15d97aa5a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:21.351327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f62d4-7256-4aca-b188-d4d15d97aa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:21.369521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e95f62d4-7256-4aca-b188-d4d15d97aa5a None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:21.370987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e95f62d4-7256-4aca-b188-d4d15d97aa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:21.378820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1210/2437] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:21.383122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc7213-6318-482d-a7f1-cd6e22a0b103 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:21.393478 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc7213-6318-482d-a7f1-cd6e22a0b103 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:21.398117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86499034-1d51-4339-93fe-2958e0625823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:21.398769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86499034-1d51-4339-93fe-2958e0625823 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:21.399050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86499034-1d51-4339-93fe-2958e0625823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:21.399326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86499034-1d51-4339-93fe-2958e0625823 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:21.400717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86499034-1d51-4339-93fe-2958e0625823 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:23:21.401171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86499034-1d51-4339-93fe-2958e0625823 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:21.402815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86499034-1d51-4339-93fe-2958e0625823 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:21.414235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1228/2438] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:23:21.418676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f62d4-7256-4aca-b188-d4d15d97aa5a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:21.428130 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f62d4-7256-4aca-b188-d4d15d97aa5a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:21.432785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a546d-5658-4369-8d96-4d486df675a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:21.433422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a546d-5658-4369-8d96-4d486df675a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:21.433716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a546d-5658-4369-8d96-4d486df675a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:21.434008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a546d-5658-4369-8d96-4d486df675a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:21.434685 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a546d-5658-4369-8d96-4d486df675a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:21.436567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a546d-5658-4369-8d96-4d486df675a1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:21.446670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:21.463548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1211/2439] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:23:21 2022] POST /identity/v3/projects => generated 409 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:23:21.467757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86499034-1d51-4339-93fe-2958e0625823 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:21.477602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86499034-1d51-4339-93fe-2958e0625823 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:21.533291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f356423-d597-49f7-a033-f7113b5ae3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:21.533993 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f356423-d597-49f7-a033-f7113b5ae3a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:21.534276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f356423-d597-49f7-a033-f7113b5ae3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:21.534549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f356423-d597-49f7-a033-f7113b5ae3a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:21.535223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f356423-d597-49f7-a033-f7113b5ae3a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:21.537146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f356423-d597-49f7-a033-f7113b5ae3a7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:21.558786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1229/2440] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:23:21 2022] POST /identity/v3/users => generated 305 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:23:21.562839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a546d-5658-4369-8d96-4d486df675a1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:21.573804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a546d-5658-4369-8d96-4d486df675a1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:21.578499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0a87f-8fc4-4603-a95a-2952b3551538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:21.579150 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0a87f-8fc4-4603-a95a-2952b3551538 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:21.579428 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0a87f-8fc4-4603-a95a-2952b3551538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:21.579700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0a87f-8fc4-4603-a95a-2952b3551538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:21.580528 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0a87f-8fc4-4603-a95a-2952b3551538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:21.582163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0a87f-8fc4-4603-a95a-2952b3551538 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:21.594368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1212/2441] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:21.598176 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f356423-d597-49f7-a033-f7113b5ae3a7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:21.608259 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f356423-d597-49f7-a033-f7113b5ae3a7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:21.613038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0584a-552b-41ad-9ce1-34243805ec03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:21.613824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0584a-552b-41ad-9ce1-34243805ec03 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:21.614106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0584a-552b-41ad-9ce1-34243805ec03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:21.614375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0584a-552b-41ad-9ce1-34243805ec03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:21.615091 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0584a-552b-41ad-9ce1-34243805ec03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:21.616686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0584a-552b-41ad-9ce1-34243805ec03 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:21.626732 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1230/2442] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:21.630653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0a87f-8fc4-4603-a95a-2952b3551538 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:21.641257 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0a87f-8fc4-4603-a95a-2952b3551538 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:21.645935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771670de-63bf-4cf5-a8c7-021cc158aea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:21.646841 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771670de-63bf-4cf5-a8c7-021cc158aea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:21.647147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771670de-63bf-4cf5-a8c7-021cc158aea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:21.647421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771670de-63bf-4cf5-a8c7-021cc158aea5 admin admin] PATH_INFO: `/v3/projects/cc6c855daf51495ab9b3bcb9dc04d8b8/users/5e09bce01e5c42a2a009fe76647af6d9/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:21.662868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771670de-63bf-4cf5-a8c7-021cc158aea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc6c855daf51495ab9b3bcb9dc04d8b8, user_id=5e09bce01e5c42a2a009fe76647af6d9, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:21.665577 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771670de-63bf-4cf5-a8c7-021cc158aea5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:21.678933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1213/2443] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:21 2022] PUT /identity/v3/projects/cc6c855daf51495ab9b3bcb9dc04d8b8/users/5e09bce01e5c42a2a009fe76647af6d9/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:23:21.689871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a9cd9-7a25-4959-aa89-ab7093e25750 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:21.690190 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a9cd9-7a25-4959-aa89-ab7093e25750 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:21.690499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a9cd9-7a25-4959-aa89-ab7093e25750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:21.711766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:21.720084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f39a9cd9-7a25-4959-aa89-ab7093e25750 None None] MFA Rules not processed for user `5e09bce01e5c42a2a009fe76647af6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:21.722092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f39a9cd9-7a25-4959-aa89-ab7093e25750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:21.785232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1231/2444] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:21 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:21.789800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771670de-63bf-4cf5-a8c7-021cc158aea5 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:21.851321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771670de-63bf-4cf5-a8c7-021cc158aea5 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:21.856062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f1443-1247-44e4-bd1b-81b950d55447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:21.856705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f1443-1247-44e4-bd1b-81b950d55447 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:21.856980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f1443-1247-44e4-bd1b-81b950d55447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:21.857250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f1443-1247-44e4-bd1b-81b950d55447 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:21.857969 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f1443-1247-44e4-bd1b-81b950d55447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:21.859860 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f1443-1247-44e4-bd1b-81b950d55447 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:21.885337 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1214/2445] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:23:21 2022] POST /identity/v3/projects => generated 409 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:23:21.890070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a9cd9-7a25-4959-aa89-ab7093e25750 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:21.900523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a9cd9-7a25-4959-aa89-ab7093e25750 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:21.959959 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abb5dc-163b-46f8-b989-b01567bfe778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:21.960629 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abb5dc-163b-46f8-b989-b01567bfe778 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:21.960909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abb5dc-163b-46f8-b989-b01567bfe778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:21.961183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abb5dc-163b-46f8-b989-b01567bfe778 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:21.961891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abb5dc-163b-46f8-b989-b01567bfe778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:21.963889 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abb5dc-163b-46f8-b989-b01567bfe778 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:21.983403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1232/2446] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:23:21 2022] POST /identity/v3/users => generated 311 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:23:21.987630 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f1443-1247-44e4-bd1b-81b950d55447 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:21.999651 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f1443-1247-44e4-bd1b-81b950d55447 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:22.004419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6aaed-250f-4d37-9a11-84dc94a19649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:22.005066 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6aaed-250f-4d37-9a11-84dc94a19649 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:22.005341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6aaed-250f-4d37-9a11-84dc94a19649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:22.005609 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6aaed-250f-4d37-9a11-84dc94a19649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:22.006352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6aaed-250f-4d37-9a11-84dc94a19649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:22.007928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6aaed-250f-4d37-9a11-84dc94a19649 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:22.019519 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1215/2447] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:22.023734 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abb5dc-163b-46f8-b989-b01567bfe778 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:22.034161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abb5dc-163b-46f8-b989-b01567bfe778 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:22.038871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccd70a-f9fd-4448-92ba-e7ed64a522d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:22.039743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccd70a-f9fd-4448-92ba-e7ed64a522d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:22.040027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccd70a-f9fd-4448-92ba-e7ed64a522d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:22.040302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccd70a-f9fd-4448-92ba-e7ed64a522d3 admin admin] PATH_INFO: `/v3/projects/14afbec52b144adbabb59168922d1fc7/users/f0feadde5b7c4424939321a9d2014e65/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:22.055537 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccd70a-f9fd-4448-92ba-e7ed64a522d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14afbec52b144adbabb59168922d1fc7, user_id=f0feadde5b7c4424939321a9d2014e65, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:22.059119 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccd70a-f9fd-4448-92ba-e7ed64a522d3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:22.071909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1233/2448] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:22 2022] PUT /identity/v3/projects/14afbec52b144adbabb59168922d1fc7/users/f0feadde5b7c4424939321a9d2014e65/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:22.076421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6aaed-250f-4d37-9a11-84dc94a19649 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:22.086418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6aaed-250f-4d37-9a11-84dc94a19649 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:22.143471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e6df5-f57e-4bfe-9472-d9c6028c4c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:22.144143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e6df5-f57e-4bfe-9472-d9c6028c4c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:22.144424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e6df5-f57e-4bfe-9472-d9c6028c4c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:22.144701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e6df5-f57e-4bfe-9472-d9c6028c4c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:22.145425 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e6df5-f57e-4bfe-9472-d9c6028c4c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:22.147184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e6df5-f57e-4bfe-9472-d9c6028c4c3c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:22.157765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1216/2449] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:22.161940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccd70a-f9fd-4448-92ba-e7ed64a522d3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:22.172875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccd70a-f9fd-4448-92ba-e7ed64a522d3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:22.177950 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc13d57-df28-4761-bcbf-a860d35d263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:22.178836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc13d57-df28-4761-bcbf-a860d35d263f admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:22.179121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc13d57-df28-4761-bcbf-a860d35d263f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:22.179403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc13d57-df28-4761-bcbf-a860d35d263f admin admin] PATH_INFO: `/v3/projects/14afbec52b144adbabb59168922d1fc7/users/f0feadde5b7c4424939321a9d2014e65/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:22.181355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc13d57-df28-4761-bcbf-a860d35d263f admin admin] RBAC: Authorizing `identity:create_grant(project_id=14afbec52b144adbabb59168922d1fc7, user_id=f0feadde5b7c4424939321a9d2014e65, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:22.184945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc13d57-df28-4761-bcbf-a860d35d263f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:22.196456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1234/2450] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:22 2022] PUT /identity/v3/projects/14afbec52b144adbabb59168922d1fc7/users/f0feadde5b7c4424939321a9d2014e65/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:23:22.207825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581720a-dcd6-4405-abbf-55241783a64b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:22.208186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581720a-dcd6-4405-abbf-55241783a64b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:22.208468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581720a-dcd6-4405-abbf-55241783a64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:22.236769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7581720a-dcd6-4405-abbf-55241783a64b None None] MFA Rules not processed for user `f0feadde5b7c4424939321a9d2014e65`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:22.238190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7581720a-dcd6-4405-abbf-55241783a64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:22.251802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:22.310624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1217/2451] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:22 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:22.321719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc13d57-df28-4761-bcbf-a860d35d263f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:22.323644 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2fc13d57-df28-4761-bcbf-a860d35d263f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:22.380552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc13d57-df28-4761-bcbf-a860d35d263f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:22.384181 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91fb9f-0d50-4c6a-befa-890f06246d7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:22.384816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91fb9f-0d50-4c6a-befa-890f06246d7b service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:22.385096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91fb9f-0d50-4c6a-befa-890f06246d7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:22.385375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91fb9f-0d50-4c6a-befa-890f06246d7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:22.392630 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:22.395473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91fb9f-0d50-4c6a-befa-890f06246d7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:22.397093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91fb9f-0d50-4c6a-befa-890f06246d7b service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:22.408148 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1235/2452] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:22 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:22.560162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581720a-dcd6-4405-abbf-55241783a64b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:22.571056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581720a-dcd6-4405-abbf-55241783a64b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:22.575594 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519ef64-7092-4900-9569-4d0d7a742f07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:22.576242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ef64-7092-4900-9569-4d0d7a742f07 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:22.576575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ef64-7092-4900-9569-4d0d7a742f07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:22.576836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ef64-7092-4900-9569-4d0d7a742f07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:22.635471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519ef64-7092-4900-9569-4d0d7a742f07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:22.637617 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519ef64-7092-4900-9569-4d0d7a742f07 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:22.646458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1218/2453] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:22 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:29.931490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91fb9f-0d50-4c6a-befa-890f06246d7b service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:29.992793 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91fb9f-0d50-4c6a-befa-890f06246d7b service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:30.004114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:30.004991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:30.005268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:30.005540 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] PATH_INFO: `/v3/users/5e09bce01e5c42a2a009fe76647af6d9` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:30.006772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e09bce01e5c42a2a009fe76647af6d9)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:30.008711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:30.054867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e09bce01e5c42a2a009fe76647af6d9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.061930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e09bce01e5c42a2a009fe76647af6d9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.062544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e09bce01e5c42a2a009fe76647af6d9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.063151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e09bce01e5c42a2a009fe76647af6d9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.075622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e09bce01e5c42a2a009fe76647af6d9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.087858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1236/2454] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:23:29 2022] DELETE /identity/v3/users/5e09bce01e5c42a2a009fe76647af6d9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:30.093474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585e452-59d6-4773-a19d-cba5a82a8023 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:30.095005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585e452-59d6-4773-a19d-cba5a82a8023 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:30.095005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585e452-59d6-4773-a19d-cba5a82a8023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:30.112001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4585e452-59d6-4773-a19d-cba5a82a8023 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:30.113227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4585e452-59d6-4773-a19d-cba5a82a8023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:30.174535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1219/2455] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:30.184852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:30.186264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:30.242861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a0a30-ac4c-45eb-8d42-5e86de50e10c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:30.247376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58aa2f-3495-4494-b067-acf2e049cf5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:30.248008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58aa2f-3495-4494-b067-acf2e049cf5f service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:30.248284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58aa2f-3495-4494-b067-acf2e049cf5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:30.248549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58aa2f-3495-4494-b067-acf2e049cf5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:30.257488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58aa2f-3495-4494-b067-acf2e049cf5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:30.258947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58aa2f-3495-4494-b067-acf2e049cf5f service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:30.268997 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1237/2456] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:30.640972 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585e452-59d6-4773-a19d-cba5a82a8023 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:30.642429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4585e452-59d6-4773-a19d-cba5a82a8023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:30.654936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585e452-59d6-4773-a19d-cba5a82a8023 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:30.665801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:30.665801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:30.665801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:30.665801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] PATH_INFO: `/v3/projects/cc6c855daf51495ab9b3bcb9dc04d8b8` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:30.666740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc6c855daf51495ab9b3bcb9dc04d8b8)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:30.666740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:30.714388 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:30.717970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc6c855daf51495ab9b3bcb9dc04d8b8', 'request_id': 'req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.725820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc6c855daf51495ab9b3bcb9dc04d8b8', 'request_id': 'req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.732879 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc6c855daf51495ab9b3bcb9dc04d8b8', 'request_id': 'req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.742970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] The token cache is being invalidate because project cc6c855daf51495ab9b3bcb9dc04d8b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:23:30.743254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.743883 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.745443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1220/2457] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:23:30 2022] DELETE /identity/v3/projects/cc6c855daf51495ab9b3bcb9dc04d8b8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:30.750232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58aa2f-3495-4494-b067-acf2e049cf5f service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:30.759832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58aa2f-3495-4494-b067-acf2e049cf5f service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:30.772047 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:30.819166 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:30.820035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:30.820320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:30.820593 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] PATH_INFO: `/v3/users/f0feadde5b7c4424939321a9d2014e65` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:30.821842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0feadde5b7c4424939321a9d2014e65)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:30.823774 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:30.874685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0feadde5b7c4424939321a9d2014e65'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.882288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0feadde5b7c4424939321a9d2014e65'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.882941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0feadde5b7c4424939321a9d2014e65'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.883545 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0feadde5b7c4424939321a9d2014e65'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.895691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0feadde5b7c4424939321a9d2014e65'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:30.908145 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1238/2458] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:23:30 2022] DELETE /identity/v3/users/f0feadde5b7c4424939321a9d2014e65 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:31.010946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:31.013351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:31.093086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883d6d4-8b0f-499a-a4c0-9a68b257d6be admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:31.157397 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184c9a6-5da3-4ce2-a536-c53f2f6cb0f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:31.158099 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184c9a6-5da3-4ce2-a536-c53f2f6cb0f9 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:31.158351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184c9a6-5da3-4ce2-a536-c53f2f6cb0f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:31.158623 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184c9a6-5da3-4ce2-a536-c53f2f6cb0f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:31.160852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d184c9a6-5da3-4ce2-a536-c53f2f6cb0f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:31.189990 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:31.191636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:31.234602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184c9a6-5da3-4ce2-a536-c53f2f6cb0f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:31.236203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184c9a6-5da3-4ce2-a536-c53f2f6cb0f9 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:31.245934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1221/2459] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:23:31.272713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eec275-40cf-4ea6-87a4-cdff503549ec admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:31.278455 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc3572-a25a-4d7a-a516-5947930265eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:31.279325 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc3572-a25a-4d7a-a516-5947930265eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:31.279604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc3572-a25a-4d7a-a516-5947930265eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:31.279874 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc3572-a25a-4d7a-a516-5947930265eb admin admin] PATH_INFO: `/v3/projects/14afbec52b144adbabb59168922d1fc7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:31.281097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc3572-a25a-4d7a-a516-5947930265eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=14afbec52b144adbabb59168922d1fc7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:31.283066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc3572-a25a-4d7a-a516-5947930265eb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:31.328692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:31.337959 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-79fc3572-a25a-4d7a-a516-5947930265eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14afbec52b144adbabb59168922d1fc7', 'request_id': 'req-79fc3572-a25a-4d7a-a516-5947930265eb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:31.346621 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-79fc3572-a25a-4d7a-a516-5947930265eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14afbec52b144adbabb59168922d1fc7', 'request_id': 'req-79fc3572-a25a-4d7a-a516-5947930265eb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:31.352519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-79fc3572-a25a-4d7a-a516-5947930265eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14afbec52b144adbabb59168922d1fc7', 'request_id': 'req-79fc3572-a25a-4d7a-a516-5947930265eb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:31.366263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-79fc3572-a25a-4d7a-a516-5947930265eb admin admin] The token cache is being invalidate because project 14afbec52b144adbabb59168922d1fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:23:31.366544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-79fc3572-a25a-4d7a-a516-5947930265eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:31.367164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-79fc3572-a25a-4d7a-a516-5947930265eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:31.369106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1239/2460] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:23:31 2022] DELETE /identity/v3/projects/14afbec52b144adbabb59168922d1fc7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:23:31.374228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2afda-fa81-42ce-aae7-9317c3c2ec10 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:31.374574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2afda-fa81-42ce-aae7-9317c3c2ec10 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:31.374914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2afda-fa81-42ce-aae7-9317c3c2ec10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:31.393507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7fe2afda-fa81-42ce-aae7-9317c3c2ec10 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:31.394658 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7fe2afda-fa81-42ce-aae7-9317c3c2ec10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:31.398984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:31.462177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1222/2461] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:31.471276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c287c4e-945f-4239-9ed9-5eacc24e5b08 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:31.471651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c287c4e-945f-4239-9ed9-5eacc24e5b08 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:31.471932 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c287c4e-945f-4239-9ed9-5eacc24e5b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:31.492953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-7c287c4e-945f-4239-9ed9-5eacc24e5b08 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:31.494219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7c287c4e-945f-4239-9ed9-5eacc24e5b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:31.506671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1240/2462] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:23:31.507689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2afda-fa81-42ce-aae7-9317c3c2ec10 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:31.518507 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2afda-fa81-42ce-aae7-9317c3c2ec10 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:31.523919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc9ac5-b00e-4fd9-b4a9-221efe7e4624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:31.524576 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc9ac5-b00e-4fd9-b4a9-221efe7e4624 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:31.524866 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc9ac5-b00e-4fd9-b4a9-221efe7e4624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:31.525120 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc9ac5-b00e-4fd9-b4a9-221efe7e4624 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:31.525881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc9ac5-b00e-4fd9-b4a9-221efe7e4624 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:23:31.526183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc9ac5-b00e-4fd9-b4a9-221efe7e4624 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:31.528863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc9ac5-b00e-4fd9-b4a9-221efe7e4624 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:31.541743 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1223/2463] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:23:31.546002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c287c4e-945f-4239-9ed9-5eacc24e5b08 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:31.556271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c287c4e-945f-4239-9ed9-5eacc24e5b08 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:31.562663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f0a68-0444-4e5b-af45-387ae1a41c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:31.564845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f0a68-0444-4e5b-af45-387ae1a41c15 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:31.564845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f0a68-0444-4e5b-af45-387ae1a41c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:31.564845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f0a68-0444-4e5b-af45-387ae1a41c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:31.565849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f0a68-0444-4e5b-af45-387ae1a41c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:31.568074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f0a68-0444-4e5b-af45-387ae1a41c15 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:31.609846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1241/2464] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:23:31 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:23:31.614274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc9ac5-b00e-4fd9-b4a9-221efe7e4624 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:31.625798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc9ac5-b00e-4fd9-b4a9-221efe7e4624 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:31.684031 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be0c07-0fe0-4803-9616-8dda62771be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:31.684733 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0c07-0fe0-4803-9616-8dda62771be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:31.685013 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0c07-0fe0-4803-9616-8dda62771be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:31.685285 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0c07-0fe0-4803-9616-8dda62771be4 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:31.686027 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be0c07-0fe0-4803-9616-8dda62771be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:31.687980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be0c07-0fe0-4803-9616-8dda62771be4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:31.709924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1224/2465] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:23:31 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:23:31.714147 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f0a68-0444-4e5b-af45-387ae1a41c15 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:31.725386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f0a68-0444-4e5b-af45-387ae1a41c15 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:31.730232 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5116d4e-5c67-4fa5-87b0-f8c663a99afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:31.730917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5116d4e-5c67-4fa5-87b0-f8c663a99afa admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:31.731183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5116d4e-5c67-4fa5-87b0-f8c663a99afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:31.731458 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5116d4e-5c67-4fa5-87b0-f8c663a99afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:31.732181 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5116d4e-5c67-4fa5-87b0-f8c663a99afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:31.733823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5116d4e-5c67-4fa5-87b0-f8c663a99afa admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:31.744326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1242/2466] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:31.749072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be0c07-0fe0-4803-9616-8dda62771be4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:31.759083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be0c07-0fe0-4803-9616-8dda62771be4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:31.764591 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5880944-ff5d-47b0-a572-f8cf8abaeaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:31.765238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5880944-ff5d-47b0-a572-f8cf8abaeaaa admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:31.765521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5880944-ff5d-47b0-a572-f8cf8abaeaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:31.765834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5880944-ff5d-47b0-a572-f8cf8abaeaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:31.766545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5880944-ff5d-47b0-a572-f8cf8abaeaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:31.768130 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5880944-ff5d-47b0-a572-f8cf8abaeaaa admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:31.778162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1225/2467] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:31.782104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5116d4e-5c67-4fa5-87b0-f8c663a99afa admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:31.792126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5116d4e-5c67-4fa5-87b0-f8c663a99afa admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:31.797016 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71791923-bf17-44ca-a35a-8532fd297ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:31.797979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71791923-bf17-44ca-a35a-8532fd297ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:31.798268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71791923-bf17-44ca-a35a-8532fd297ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:31.798550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71791923-bf17-44ca-a35a-8532fd297ebe admin admin] PATH_INFO: `/v3/projects/6a6841e8ca214315a5516d5339d57003/users/a99095a5d67e4cd98c1a77b841df9d32/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:31.814561 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71791923-bf17-44ca-a35a-8532fd297ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6841e8ca214315a5516d5339d57003, user_id=a99095a5d67e4cd98c1a77b841df9d32, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:31.817262 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71791923-bf17-44ca-a35a-8532fd297ebe admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:31.830669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1243/2468] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:31 2022] PUT /identity/v3/projects/6a6841e8ca214315a5516d5339d57003/users/a99095a5d67e4cd98c1a77b841df9d32/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:23:31.841187 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf92bb-459d-4ab4-ba75-1e301596ddf0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:31.841478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf92bb-459d-4ab4-ba75-1e301596ddf0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:31.841785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf92bb-459d-4ab4-ba75-1e301596ddf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:31.868896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-afaf92bb-459d-4ab4-ba75-1e301596ddf0 None None] MFA Rules not processed for user `a99095a5d67e4cd98c1a77b841df9d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:31.870143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-afaf92bb-459d-4ab4-ba75-1e301596ddf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:31.933071 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1226/2469] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:31.945040 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71791923-bf17-44ca-a35a-8532fd297ebe admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:31.946791 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-71791923-bf17-44ca-a35a-8532fd297ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:32.003823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71791923-bf17-44ca-a35a-8532fd297ebe admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:32.007378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75dbb6-1f1f-443b-b7b3-3eded7ed27c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:32.008009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75dbb6-1f1f-443b-b7b3-3eded7ed27c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:32.008285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75dbb6-1f1f-443b-b7b3-3eded7ed27c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:32.008555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75dbb6-1f1f-443b-b7b3-3eded7ed27c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:32.018748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75dbb6-1f1f-443b-b7b3-3eded7ed27c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:32.020258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75dbb6-1f1f-443b-b7b3-3eded7ed27c0 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:32.029052 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1244/2470] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:37.298473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4acf3b-1cdf-480f-82c6-ae99807a76f6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:37.298954 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4acf3b-1cdf-480f-82c6-ae99807a76f6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:37.299072 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4acf3b-1cdf-480f-82c6-ae99807a76f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:37.306188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:37.318934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4c4acf3b-1cdf-480f-82c6-ae99807a76f6 None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:37.320155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4c4acf3b-1cdf-480f-82c6-ae99807a76f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:37.378676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1227/2471] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:23:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:37.391119 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75dbb6-1f1f-443b-b7b3-3eded7ed27c0 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:37.392606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ba75dbb6-1f1f-443b-b7b3-3eded7ed27c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:37.471895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75dbb6-1f1f-443b-b7b3-3eded7ed27c0 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:37.476914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dc34a-8e35-4768-b4f0-dded0c0ed2d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:37.477685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dc34a-8e35-4768-b4f0-dded0c0ed2d6 service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:37.478002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dc34a-8e35-4768-b4f0-dded0c0ed2d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:37.478280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dc34a-8e35-4768-b4f0-dded0c0ed2d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:37.485417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:37.488155 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dc34a-8e35-4768-b4f0-dded0c0ed2d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:37.490500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dc34a-8e35-4768-b4f0-dded0c0ed2d6 service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:37.502304 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1245/2472] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:23:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:23:39.159208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4acf3b-1cdf-480f-82c6-ae99807a76f6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:39.163598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4c4acf3b-1cdf-480f-82c6-ae99807a76f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:39.267900 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4acf3b-1cdf-480f-82c6-ae99807a76f6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:39.274088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bc5cf-a013-4963-ae0c-6ae00351826b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:39.274830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bc5cf-a013-4963-ae0c-6ae00351826b service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:39.275181 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bc5cf-a013-4963-ae0c-6ae00351826b service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:39.275700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bc5cf-a013-4963-ae0c-6ae00351826b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:39.278019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-311bc5cf-a013-4963-ae0c-6ae00351826b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:39.293769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bc5cf-a013-4963-ae0c-6ae00351826b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:39.295467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bc5cf-a013-4963-ae0c-6ae00351826b service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:39.308426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1228/2473] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:47.498940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:49.303498 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=57 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:51.933499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dc34a-8e35-4768-b4f0-dded0c0ed2d6 service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:51.998833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dc34a-8e35-4768-b4f0-dded0c0ed2d6 service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:52.007316 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:52.008413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:52.008978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:52.009377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] PATH_INFO: `/v3/users/a99095a5d67e4cd98c1a77b841df9d32` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:52.011279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a99095a5d67e4cd98c1a77b841df9d32)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:52.018186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:52.088676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a99095a5d67e4cd98c1a77b841df9d32'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:52.102589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a99095a5d67e4cd98c1a77b841df9d32'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:52.103239 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a99095a5d67e4cd98c1a77b841df9d32'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:52.104224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a99095a5d67e4cd98c1a77b841df9d32'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:52.134984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a99095a5d67e4cd98c1a77b841df9d32'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:52.159006 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1246/2474] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:23:51 2022] DELETE /identity/v3/users/a99095a5d67e4cd98c1a77b841df9d32 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:52.166661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef04339-c97e-45a8-8f4b-da42631613d7 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:52.167142 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef04339-c97e-45a8-8f4b-da42631613d7 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:52.167240 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef04339-c97e-45a8-8f4b-da42631613d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:52.186784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4ef04339-c97e-45a8-8f4b-da42631613d7 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:52.188011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4ef04339-c97e-45a8-8f4b-da42631613d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:52.248211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1229/2475] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:52.258194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:52.259597 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:52.327089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db8ac7-9152-4a0b-83ed-167dc8671bf3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:52.330865 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71206646-e3e2-42ec-917f-6e8bc93b675e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:52.331588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71206646-e3e2-42ec-917f-6e8bc93b675e service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:52.331970 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71206646-e3e2-42ec-917f-6e8bc93b675e service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:52.332314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71206646-e3e2-42ec-917f-6e8bc93b675e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:52.343436 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71206646-e3e2-42ec-917f-6e8bc93b675e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:52.345231 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71206646-e3e2-42ec-917f-6e8bc93b675e service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:52.357417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1247/2476] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:52.645204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef04339-c97e-45a8-8f4b-da42631613d7 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:52.646716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4ef04339-c97e-45a8-8f4b-da42631613d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:52.664560 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef04339-c97e-45a8-8f4b-da42631613d7 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:52.672089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20117b61-b54b-4efe-955f-11df65150f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:52.673229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20117b61-b54b-4efe-955f-11df65150f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:52.673513 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20117b61-b54b-4efe-955f-11df65150f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:52.673815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20117b61-b54b-4efe-955f-11df65150f58 admin admin] PATH_INFO: `/v3/projects/6a6841e8ca214315a5516d5339d57003` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:52.675087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20117b61-b54b-4efe-955f-11df65150f58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a6841e8ca214315a5516d5339d57003)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:52.677275 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20117b61-b54b-4efe-955f-11df65150f58 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:52.730554 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-20117b61-b54b-4efe-955f-11df65150f58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a6841e8ca214315a5516d5339d57003', 'request_id': 'req-20117b61-b54b-4efe-955f-11df65150f58'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:52.738665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-20117b61-b54b-4efe-955f-11df65150f58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a6841e8ca214315a5516d5339d57003', 'request_id': 'req-20117b61-b54b-4efe-955f-11df65150f58'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:52.746393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-20117b61-b54b-4efe-955f-11df65150f58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a6841e8ca214315a5516d5339d57003', 'request_id': 'req-20117b61-b54b-4efe-955f-11df65150f58'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:52.759888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-20117b61-b54b-4efe-955f-11df65150f58 admin admin] The token cache is being invalidate because project 6a6841e8ca214315a5516d5339d57003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:23:52.760173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-20117b61-b54b-4efe-955f-11df65150f58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:52.760846 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-20117b61-b54b-4efe-955f-11df65150f58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:23:52.763175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1230/2477] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:23:52 2022] DELETE /identity/v3/projects/6a6841e8ca214315a5516d5339d57003 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:52.768318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e50e0-b326-4e47-925f-5cc17be24257 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:52.768619 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e50e0-b326-4e47-925f-5cc17be24257 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:52.768988 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e50e0-b326-4e47-925f-5cc17be24257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:52.787522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-454e50e0-b326-4e47-925f-5cc17be24257 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:52.788760 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-454e50e0-b326-4e47-925f-5cc17be24257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:52.807084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:52.854390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1248/2478] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:52.863825 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24926b-70bb-4620-969b-711388c45209 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:52.864115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24926b-70bb-4620-969b-711388c45209 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:52.864426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24926b-70bb-4620-969b-711388c45209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:52.883293 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4a24926b-70bb-4620-969b-711388c45209 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:52.884499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4a24926b-70bb-4620-969b-711388c45209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:52.893056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1231/2479] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:52.897579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e50e0-b326-4e47-925f-5cc17be24257 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:52.911161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e50e0-b326-4e47-925f-5cc17be24257 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:52.915835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830ef3f-49e9-4fcf-81fa-b04bfd0bc246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:52.916482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830ef3f-49e9-4fcf-81fa-b04bfd0bc246 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:52.916763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830ef3f-49e9-4fcf-81fa-b04bfd0bc246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:52.917034 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830ef3f-49e9-4fcf-81fa-b04bfd0bc246 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:52.917779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830ef3f-49e9-4fcf-81fa-b04bfd0bc246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:23:52.918088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830ef3f-49e9-4fcf-81fa-b04bfd0bc246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:52.920247 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830ef3f-49e9-4fcf-81fa-b04bfd0bc246 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:52.936103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1249/2480] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:23:52.937354 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24926b-70bb-4620-969b-711388c45209 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:52.947299 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24926b-70bb-4620-969b-711388c45209 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:52.952845 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5dea0-4f93-46a0-be28-d52631a07b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:52.953646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5dea0-4f93-46a0-be28-d52631a07b03 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:52.954024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5dea0-4f93-46a0-be28-d52631a07b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:52.954369 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5dea0-4f93-46a0-be28-d52631a07b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:52.955131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5dea0-4f93-46a0-be28-d52631a07b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:52.957115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5dea0-4f93-46a0-be28-d52631a07b03 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:53.000946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1232/2481] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:23:52 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:23:53.002234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830ef3f-49e9-4fcf-81fa-b04bfd0bc246 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:53.003845 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5dea0-4f93-46a0-be28-d52631a07b03 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:53.005434 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-39f5dea0-4f93-46a0-be28-d52631a07b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:53.012850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830ef3f-49e9-4fcf-81fa-b04bfd0bc246 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:53.062145 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:53.067459 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5dea0-4f93-46a0-be28-d52631a07b03 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:53.073205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbbe41-f213-4902-9874-e19a99e30d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:53.074219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbbe41-f213-4902-9874-e19a99e30d09 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.074810 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbbe41-f213-4902-9874-e19a99e30d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.075740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbbe41-f213-4902-9874-e19a99e30d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.078652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a93b78-23b7-4817-9e97-f892ebdb53ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:53.078832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-eecbbe41-f213-4902-9874-e19a99e30d09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:53.079328 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a93b78-23b7-4817-9e97-f892ebdb53ce admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.079615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a93b78-23b7-4817-9e97-f892ebdb53ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.079892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a93b78-23b7-4817-9e97-f892ebdb53ce admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.080578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a93b78-23b7-4817-9e97-f892ebdb53ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:53.082553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a93b78-23b7-4817-9e97-f892ebdb53ce admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:53.089358 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbbe41-f213-4902-9874-e19a99e30d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:53.091668 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbbe41-f213-4902-9874-e19a99e30d09 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:53.100932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1233/2482] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:53.105774 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1250/2483] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:23:53 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:23:53.114038 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbbe41-f213-4902-9874-e19a99e30d09 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:53.129795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbbe41-f213-4902-9874-e19a99e30d09 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:53.135058 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2e945-d986-4e84-9098-7e027c4778f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:53.136223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2e945-d986-4e84-9098-7e027c4778f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.136690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2e945-d986-4e84-9098-7e027c4778f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.137142 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2e945-d986-4e84-9098-7e027c4778f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.138579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2e945-d986-4e84-9098-7e027c4778f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:53.141082 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2e945-d986-4e84-9098-7e027c4778f3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:53.155678 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1234/2484] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:23:53.160144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a93b78-23b7-4817-9e97-f892ebdb53ce admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:53.181118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a93b78-23b7-4817-9e97-f892ebdb53ce admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:53.191670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15b79b-afa2-4b21-95bb-bdffe0f82cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:53.192449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15b79b-afa2-4b21-95bb-bdffe0f82cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.192722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15b79b-afa2-4b21-95bb-bdffe0f82cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.193032 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15b79b-afa2-4b21-95bb-bdffe0f82cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.193823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15b79b-afa2-4b21-95bb-bdffe0f82cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:53.195566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15b79b-afa2-4b21-95bb-bdffe0f82cc4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:53.209564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1251/2485] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:23:53.214859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2e945-d986-4e84-9098-7e027c4778f3 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:53.228263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2e945-d986-4e84-9098-7e027c4778f3 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:53.231690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874ed2f-c66c-4059-a9c5-d667bf0ee112 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.232112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874ed2f-c66c-4059-a9c5-d667bf0ee112 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.232491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874ed2f-c66c-4059-a9c5-d667bf0ee112 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.234281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1252/2486] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:23:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:23:53.239605 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d8800-d3b4-412c-88ea-600fdb9567fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:53.240522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47099a-28a3-4762-968b-0f2e2af7cbeb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.240780 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d8800-d3b4-412c-88ea-600fdb9567fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.240894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47099a-28a3-4762-968b-0f2e2af7cbeb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.241237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d8800-d3b4-412c-88ea-600fdb9567fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.241355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47099a-28a3-4762-968b-0f2e2af7cbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.241687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d8800-d3b4-412c-88ea-600fdb9567fe admin admin] PATH_INFO: `/v3/projects/80708cb25d834c8fa35f02a0ce76442a/users/de1afd71132a40e2a3a2a985fbd5b45b/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.259176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d8800-d3b4-412c-88ea-600fdb9567fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=80708cb25d834c8fa35f02a0ce76442a, user_id=de1afd71132a40e2a3a2a985fbd5b45b, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:53.262906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d8800-d3b4-412c-88ea-600fdb9567fe admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:53.265516 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1b47099a-28a3-4762-968b-0f2e2af7cbeb None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:53.266957 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1b47099a-28a3-4762-968b-0f2e2af7cbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:53.283616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1235/2487] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:23:53 2022] PUT /identity/v3/projects/80708cb25d834c8fa35f02a0ce76442a/users/de1afd71132a40e2a3a2a985fbd5b45b/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:53.293909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89d6b0-e46b-48a4-b3ee-32b96c48b13f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.294863 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89d6b0-e46b-48a4-b3ee-32b96c48b13f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.295141 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89d6b0-e46b-48a4-b3ee-32b96c48b13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.328137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4f89d6b0-e46b-48a4-b3ee-32b96c48b13f None None] MFA Rules not processed for user `de1afd71132a40e2a3a2a985fbd5b45b`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:53.329639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4f89d6b0-e46b-48a4-b3ee-32b96c48b13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:53.360427 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1253/2488] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:23:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:23:53.373669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47099a-28a3-4762-968b-0f2e2af7cbeb None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:53.375046 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1b47099a-28a3-4762-968b-0f2e2af7cbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:53.408505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:53.409291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1236/2489] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:23:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:23:53.424779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89d6b0-e46b-48a4-b3ee-32b96c48b13f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:53.424779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4f89d6b0-e46b-48a4-b3ee-32b96c48b13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:53.451880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47099a-28a3-4762-968b-0f2e2af7cbeb None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:53.488900 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89d6b0-e46b-48a4-b3ee-32b96c48b13f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:53.492704 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81313059-6a37-4c0d-a96e-9b060962d0ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:53.493353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81313059-6a37-4c0d-a96e-9b060962d0ba service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.493633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81313059-6a37-4c0d-a96e-9b060962d0ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.493937 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81313059-6a37-4c0d-a96e-9b060962d0ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.503476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81313059-6a37-4c0d-a96e-9b060962d0ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:53.504909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81313059-6a37-4c0d-a96e-9b060962d0ba service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:53.509867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e036ce-e141-47a6-b524-b162eea7dc56 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a67120485f74f8bb0ff188cd3abd21c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:53.510523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e036ce-e141-47a6-b524-b162eea7dc56 service glance] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.510802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e036ce-e141-47a6-b524-b162eea7dc56 service glance] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.511078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e036ce-e141-47a6-b524-b162eea7dc56 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.515225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1237/2490] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:53.611501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e036ce-e141-47a6-b524-b162eea7dc56 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:53.613104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e036ce-e141-47a6-b524-b162eea7dc56 service glance] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:53.627483 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1254/2491] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:53.738702 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb23683-0a52-479c-94f5-d946c4baf3cc None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.739420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb23683-0a52-479c-94f5-d946c4baf3cc None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.739851 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb23683-0a52-479c-94f5-d946c4baf3cc None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.743470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1238/2492] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:23:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:23:53.749390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec9eda-ffd7-42e8-96f2-a42c73107e6a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.749681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec9eda-ffd7-42e8-96f2-a42c73107e6a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.750101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec9eda-ffd7-42e8-96f2-a42c73107e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.771852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-19ec9eda-ffd7-42e8-96f2-a42c73107e6a None None] MFA Rules not processed for user `f668c4c6745e4f1eae20d8a3a12cf0cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:53.773153 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-19ec9eda-ffd7-42e8-96f2-a42c73107e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:53.830722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1255/2493] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:23:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:53.843800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb23683-0a52-479c-94f5-d946c4baf3cc None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:53.845758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-fbb23683-0a52-479c-94f5-d946c4baf3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:53.861303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb23683-0a52-479c-94f5-d946c4baf3cc None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:53.865471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6060c8-d33e-4985-a127-e5a58904cdac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:53.866145 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6060c8-d33e-4985-a127-e5a58904cdac service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.866419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6060c8-d33e-4985-a127-e5a58904cdac service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.866688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6060c8-d33e-4985-a127-e5a58904cdac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.874653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6060c8-d33e-4985-a127-e5a58904cdac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:53.876078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6060c8-d33e-4985-a127-e5a58904cdac service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:53.889028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1239/2494] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:53.983998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2082fcc-2926-4846-85bb-1fb5e4db011f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.984499 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2082fcc-2926-4846-85bb-1fb5e4db011f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.984613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2082fcc-2926-4846-85bb-1fb5e4db011f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:53.986069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1256/2495] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 18:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:23:53.999102 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce1692-31c5-4c0c-a556-70af30b5ff4e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:53.999414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce1692-31c5-4c0c-a556-70af30b5ff4e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:53.999689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce1692-31c5-4c0c-a556-70af30b5ff4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:54.018598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-36ce1692-31c5-4c0c-a556-70af30b5ff4e None None] MFA Rules not processed for user `d420e808730d484f80963322c5fc6856`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:54.019797 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-36ce1692-31c5-4c0c-a556-70af30b5ff4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:54.031529 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1240/2496] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 18:23:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:54.041845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2082fcc-2926-4846-85bb-1fb5e4db011f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:54.052792 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2082fcc-2926-4846-85bb-1fb5e4db011f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:54.056273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396dacd-c57c-4626-91be-a983d0e3de2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:54.056894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396dacd-c57c-4626-91be-a983d0e3de2c service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:54.057172 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396dacd-c57c-4626-91be-a983d0e3de2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:54.057444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396dacd-c57c-4626-91be-a983d0e3de2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:54.067957 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396dacd-c57c-4626-91be-a983d0e3de2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:54.069386 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396dacd-c57c-4626-91be-a983d0e3de2c service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:54.080479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1257/2497] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:23:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:55.635170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20784725-6604-4bc0-a081-e4fe1b8a8408 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:55.635636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20784725-6604-4bc0-a081-e4fe1b8a8408 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:55.635752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20784725-6604-4bc0-a081-e4fe1b8a8408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:55.655567 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-20784725-6604-4bc0-a081-e4fe1b8a8408 None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:23:55.656784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-20784725-6604-4bc0-a081-e4fe1b8a8408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:55.680415 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:55.720833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1241/2498] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:23:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:23:55.732762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396dacd-c57c-4626-91be-a983d0e3de2c service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:55.734735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d396dacd-c57c-4626-91be-a983d0e3de2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:55.749908 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:55.805084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396dacd-c57c-4626-91be-a983d0e3de2c service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:55.811234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6ccaf-3300-4e8d-9ec9-f99b976413ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:55.811883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6ccaf-3300-4e8d-9ec9-f99b976413ef service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:55.812167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6ccaf-3300-4e8d-9ec9-f99b976413ef service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:55.812448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6ccaf-3300-4e8d-9ec9-f99b976413ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:55.820697 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6ccaf-3300-4e8d-9ec9-f99b976413ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:55.822485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6ccaf-3300-4e8d-9ec9-f99b976413ef service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:55.832228 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1258/2499] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:23:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:24:01.995612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20784725-6604-4bc0-a081-e4fe1b8a8408 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:02.006105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20784725-6604-4bc0-a081-e4fe1b8a8408 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:02.010791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f962ca-06da-4629-8af7-76305d65fb46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:02.011516 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f962ca-06da-4629-8af7-76305d65fb46 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:02.011962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f962ca-06da-4629-8af7-76305d65fb46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:02.012371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f962ca-06da-4629-8af7-76305d65fb46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:02.015076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-56f962ca-06da-4629-8af7-76305d65fb46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:02.090167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f962ca-06da-4629-8af7-76305d65fb46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:02.091924 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f962ca-06da-4629-8af7-76305d65fb46 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:02.103836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1242/2500] 10.222.0.46 () {60 vars in 1396 bytes} [Thu May 26 18:24:01 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:03.680754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6ccaf-3300-4e8d-9ec9-f99b976413ef service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:03.743720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6ccaf-3300-4e8d-9ec9-f99b976413ef service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:03.748369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:03.749236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:03.749521 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:03.749838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] PATH_INFO: `/v3/users/de1afd71132a40e2a3a2a985fbd5b45b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:03.751052 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1afd71132a40e2a3a2a985fbd5b45b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:03.752988 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:03.800035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1afd71132a40e2a3a2a985fbd5b45b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:03.807510 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1afd71132a40e2a3a2a985fbd5b45b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:03.808144 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1afd71132a40e2a3a2a985fbd5b45b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:03.808748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1afd71132a40e2a3a2a985fbd5b45b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:03.820928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1afd71132a40e2a3a2a985fbd5b45b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:03.833453 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1259/2501] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:03 2022] DELETE /identity/v3/users/de1afd71132a40e2a3a2a985fbd5b45b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:03.838006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f482b-8073-4070-ab9e-a0cf2f62a6e6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:03.838494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f482b-8073-4070-ab9e-a0cf2f62a6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:03.838599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f482b-8073-4070-ab9e-a0cf2f62a6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:03.857100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ca1f482b-8073-4070-ab9e-a0cf2f62a6e6 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:03.858265 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ca1f482b-8073-4070-ab9e-a0cf2f62a6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:03.916327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1243/2502] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:03.926768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:03.928261 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:03.954630 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:03.987449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583efcb-88c9-4212-8ed1-11f5c249dddf admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:03.991059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c184493-4a64-497c-b71e-32a4cb4824be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:03.991698 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c184493-4a64-497c-b71e-32a4cb4824be service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:03.991978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c184493-4a64-497c-b71e-32a4cb4824be service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:03.992251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c184493-4a64-497c-b71e-32a4cb4824be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:04.001627 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c184493-4a64-497c-b71e-32a4cb4824be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:04.003116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c184493-4a64-497c-b71e-32a4cb4824be service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:04.013645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1260/2503] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:04.287593 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f482b-8073-4070-ab9e-a0cf2f62a6e6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:04.289217 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ca1f482b-8073-4070-ab9e-a0cf2f62a6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:04.310409 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f482b-8073-4070-ab9e-a0cf2f62a6e6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:04.319738 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360ed4a-0bd4-4326-9413-6a28868338e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:04.320614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360ed4a-0bd4-4326-9413-6a28868338e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:04.320894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360ed4a-0bd4-4326-9413-6a28868338e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:04.321612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360ed4a-0bd4-4326-9413-6a28868338e4 admin admin] PATH_INFO: `/v3/projects/80708cb25d834c8fa35f02a0ce76442a` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:04.323193 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360ed4a-0bd4-4326-9413-6a28868338e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80708cb25d834c8fa35f02a0ce76442a)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:04.325836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360ed4a-0bd4-4326-9413-6a28868338e4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:04.378849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1360ed4a-0bd4-4326-9413-6a28868338e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80708cb25d834c8fa35f02a0ce76442a', 'request_id': 'req-1360ed4a-0bd4-4326-9413-6a28868338e4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:04.386075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1360ed4a-0bd4-4326-9413-6a28868338e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80708cb25d834c8fa35f02a0ce76442a', 'request_id': 'req-1360ed4a-0bd4-4326-9413-6a28868338e4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:04.392380 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1360ed4a-0bd4-4326-9413-6a28868338e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80708cb25d834c8fa35f02a0ce76442a', 'request_id': 'req-1360ed4a-0bd4-4326-9413-6a28868338e4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:04.396303 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:04.407420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1360ed4a-0bd4-4326-9413-6a28868338e4 admin admin] The token cache is being invalidate because project 80708cb25d834c8fa35f02a0ce76442a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:04.408131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1360ed4a-0bd4-4326-9413-6a28868338e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:04.408628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1360ed4a-0bd4-4326-9413-6a28868338e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:04.411376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1244/2504] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:04 2022] DELETE /identity/v3/projects/80708cb25d834c8fa35f02a0ce76442a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:10.485606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c184493-4a64-497c-b71e-32a4cb4824be service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:10.487108 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1c184493-4a64-497c-b71e-32a4cb4824be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:10.565010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c184493-4a64-497c-b71e-32a4cb4824be service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:10.643575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0e008-987c-4da5-92bd-b402fffa50a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:10.644237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0e008-987c-4da5-92bd-b402fffa50a3 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:10.644520 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0e008-987c-4da5-92bd-b402fffa50a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:10.644795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0e008-987c-4da5-92bd-b402fffa50a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:10.646623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-fbc0e008-987c-4da5-92bd-b402fffa50a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:10.660001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc0e008-987c-4da5-92bd-b402fffa50a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:10.661624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc0e008-987c-4da5-92bd-b402fffa50a3 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:10.673924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1261/2505] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:12.025498 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb270e-f6d3-4645-9dfb-3e36f1b8bf4f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:12.026000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb270e-f6d3-4645-9dfb-3e36f1b8bf4f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:12.026115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb270e-f6d3-4645-9dfb-3e36f1b8bf4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:12.049674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-46eb270e-f6d3-4645-9dfb-3e36f1b8bf4f None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:12.054541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-46eb270e-f6d3-4645-9dfb-3e36f1b8bf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:12.143265 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1245/2506] 10.222.0.46 () {58 vars in 967 bytes} [Thu May 26 18:24:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:12.149676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0e008-987c-4da5-92bd-b402fffa50a3 service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:12.151840 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-fbc0e008-987c-4da5-92bd-b402fffa50a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:12.213812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:12.248722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0e008-987c-4da5-92bd-b402fffa50a3 service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:12.254115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bddeeb-a130-4b5b-a99e-35f9b267f372 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:12.254772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bddeeb-a130-4b5b-a99e-35f9b267f372 service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:12.255061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bddeeb-a130-4b5b-a99e-35f9b267f372 service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:12.255334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bddeeb-a130-4b5b-a99e-35f9b267f372 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:12.264085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bddeeb-a130-4b5b-a99e-35f9b267f372 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:12.265714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bddeeb-a130-4b5b-a99e-35f9b267f372 service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:12.280767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1262/2507] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:24:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:24:17.587637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb270e-f6d3-4645-9dfb-3e36f1b8bf4f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:17.589215 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-46eb270e-f6d3-4645-9dfb-3e36f1b8bf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:17.650492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb270e-f6d3-4645-9dfb-3e36f1b8bf4f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:17.654894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75b41f-c1df-425c-8c43-6033651f948a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:17.655580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75b41f-c1df-425c-8c43-6033651f948a service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:17.655896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75b41f-c1df-425c-8c43-6033651f948a service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:17.656133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75b41f-c1df-425c-8c43-6033651f948a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:17.657958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3e75b41f-c1df-425c-8c43-6033651f948a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:17.726209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75b41f-c1df-425c-8c43-6033651f948a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:17.727885 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75b41f-c1df-425c-8c43-6033651f948a service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:17.738281 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1246/2508] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:18.826183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bddeeb-a130-4b5b-a99e-35f9b267f372 service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:18.827635 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-00bddeeb-a130-4b5b-a99e-35f9b267f372 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:18.839625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bddeeb-a130-4b5b-a99e-35f9b267f372 service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:18.843885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9de2b8-79a5-4b54-bc7d-239765a6e256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:18.844531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9de2b8-79a5-4b54-bc7d-239765a6e256 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:18.844811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9de2b8-79a5-4b54-bc7d-239765a6e256 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:18.845082 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9de2b8-79a5-4b54-bc7d-239765a6e256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:18.846891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-fb9de2b8-79a5-4b54-bc7d-239765a6e256 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:18.858897 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9de2b8-79a5-4b54-bc7d-239765a6e256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:18.860342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9de2b8-79a5-4b54-bc7d-239765a6e256 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:18.870802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1263/2509] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:21.150528 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75b41f-c1df-425c-8c43-6033651f948a service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:21.154028 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3e75b41f-c1df-425c-8c43-6033651f948a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:21.176959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75b41f-c1df-425c-8c43-6033651f948a service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:21.183052 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e225b-14a7-4386-9a09-995b4ce68cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:21.184948 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e225b-14a7-4386-9a09-995b4ce68cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:21.185492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e225b-14a7-4386-9a09-995b4ce68cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:21.186256 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e225b-14a7-4386-9a09-995b4ce68cf0 admin admin] PATH_INFO: `/v3/users/8da0a5a09a6041d68c5e445af1a492f3` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:21.187779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e225b-14a7-4386-9a09-995b4ce68cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8da0a5a09a6041d68c5e445af1a492f3)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:21.190069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e225b-14a7-4386-9a09-995b4ce68cf0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:21.249461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=7,keystone:SELECT=93 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:21.252899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c93e225b-14a7-4386-9a09-995b4ce68cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8da0a5a09a6041d68c5e445af1a492f3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:21.263641 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c93e225b-14a7-4386-9a09-995b4ce68cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8da0a5a09a6041d68c5e445af1a492f3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:21.283077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c93e225b-14a7-4386-9a09-995b4ce68cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8da0a5a09a6041d68c5e445af1a492f3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:21.283817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c93e225b-14a7-4386-9a09-995b4ce68cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8da0a5a09a6041d68c5e445af1a492f3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:21.292172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c93e225b-14a7-4386-9a09-995b4ce68cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8da0a5a09a6041d68c5e445af1a492f3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:21.302773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1247/2510] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:21 2022] DELETE /identity/v3/users/8da0a5a09a6041d68c5e445af1a492f3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:21.474705 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9de2b8-79a5-4b54-bc7d-239765a6e256 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:21.496288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9de2b8-79a5-4b54-bc7d-239765a6e256 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:21.556990 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b1b29-3636-4ef8-9170-f66c18652bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:21.557883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b1b29-3636-4ef8-9170-f66c18652bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:21.558161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b1b29-3636-4ef8-9170-f66c18652bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:21.558427 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b1b29-3636-4ef8-9170-f66c18652bf6 admin admin] PATH_INFO: `/v3/projects/31b4afa4107c4cd2ac0212c45d998801` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:21.560099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b1b29-3636-4ef8-9170-f66c18652bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b4afa4107c4cd2ac0212c45d998801)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:21.562061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b1b29-3636-4ef8-9170-f66c18652bf6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:21.613780 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-364b1b29-3636-4ef8-9170-f66c18652bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b4afa4107c4cd2ac0212c45d998801', 'request_id': 'req-364b1b29-3636-4ef8-9170-f66c18652bf6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:21.621491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-364b1b29-3636-4ef8-9170-f66c18652bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b4afa4107c4cd2ac0212c45d998801', 'request_id': 'req-364b1b29-3636-4ef8-9170-f66c18652bf6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:21.628831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-364b1b29-3636-4ef8-9170-f66c18652bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b4afa4107c4cd2ac0212c45d998801', 'request_id': 'req-364b1b29-3636-4ef8-9170-f66c18652bf6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:21.645569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-364b1b29-3636-4ef8-9170-f66c18652bf6 admin admin] The token cache is being invalidate because project 31b4afa4107c4cd2ac0212c45d998801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:21.646552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-364b1b29-3636-4ef8-9170-f66c18652bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:21.647448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-364b1b29-3636-4ef8-9170-f66c18652bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:21.649021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1264/2511] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:21 2022] DELETE /identity/v3/projects/31b4afa4107c4cd2ac0212c45d998801 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:21.654911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf52f45-6282-4b43-a044-ed272f25545a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:21.655200 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf52f45-6282-4b43-a044-ed272f25545a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:21.655674 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf52f45-6282-4b43-a044-ed272f25545a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:21.675893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4bf52f45-6282-4b43-a044-ed272f25545a None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:21.678503 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4bf52f45-6282-4b43-a044-ed272f25545a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:21.759284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1248/2512] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:21.767522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81aefa-e2c2-41fa-9d0e-c1b927e5fa94 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:21.767898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81aefa-e2c2-41fa-9d0e-c1b927e5fa94 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:21.768250 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81aefa-e2c2-41fa-9d0e-c1b927e5fa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:21.789469 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8e81aefa-e2c2-41fa-9d0e-c1b927e5fa94 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:21.790850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8e81aefa-e2c2-41fa-9d0e-c1b927e5fa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:21.800831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1265/2513] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:21.804510 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf52f45-6282-4b43-a044-ed272f25545a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:21.816758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf52f45-6282-4b43-a044-ed272f25545a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:21.826411 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1980c7-7ce8-48f2-8dcc-97afa41ac596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:21.827111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1980c7-7ce8-48f2-8dcc-97afa41ac596 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:21.827426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1980c7-7ce8-48f2-8dcc-97afa41ac596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:21.827899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1980c7-7ce8-48f2-8dcc-97afa41ac596 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:21.829046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1980c7-7ce8-48f2-8dcc-97afa41ac596 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:21.829363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1980c7-7ce8-48f2-8dcc-97afa41ac596 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:21.843611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1980c7-7ce8-48f2-8dcc-97afa41ac596 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:21.860555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1249/2514] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:24:21.865075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81aefa-e2c2-41fa-9d0e-c1b927e5fa94 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:21.877061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81aefa-e2c2-41fa-9d0e-c1b927e5fa94 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:21.883984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac0ce9-d194-4d7a-97c4-882a95344b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:21.884680 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac0ce9-d194-4d7a-97c4-882a95344b39 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:21.884956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac0ce9-d194-4d7a-97c4-882a95344b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:21.885227 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac0ce9-d194-4d7a-97c4-882a95344b39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:21.885995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac0ce9-d194-4d7a-97c4-882a95344b39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:21.888831 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac0ce9-d194-4d7a-97c4-882a95344b39 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:21.924579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1266/2515] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:21 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:21.928366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1980c7-7ce8-48f2-8dcc-97afa41ac596 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:21.939685 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1980c7-7ce8-48f2-8dcc-97afa41ac596 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:22.009091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529eef21-fc89-482d-9ee9-56c5e53810ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:22.009889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529eef21-fc89-482d-9ee9-56c5e53810ab admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.010171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529eef21-fc89-482d-9ee9-56c5e53810ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.010443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529eef21-fc89-482d-9ee9-56c5e53810ab admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.011131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529eef21-fc89-482d-9ee9-56c5e53810ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:22.013820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529eef21-fc89-482d-9ee9-56c5e53810ab admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:22.040061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1250/2516] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:21 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:22.043995 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac0ce9-d194-4d7a-97c4-882a95344b39 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:22.068064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac0ce9-d194-4d7a-97c4-882a95344b39 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:22.076109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e68b6f-8768-4d38-9692-94d9123632fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:22.077059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e68b6f-8768-4d38-9692-94d9123632fa admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.077346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e68b6f-8768-4d38-9692-94d9123632fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.078106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e68b6f-8768-4d38-9692-94d9123632fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.079446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e68b6f-8768-4d38-9692-94d9123632fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:22.084957 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e68b6f-8768-4d38-9692-94d9123632fa admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:22.110523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1267/2517] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:22.111817 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529eef21-fc89-482d-9ee9-56c5e53810ab admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:22.126534 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529eef21-fc89-482d-9ee9-56c5e53810ab admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:22.134583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fc07d-0483-4574-9d73-904a703e8461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:22.135416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fc07d-0483-4574-9d73-904a703e8461 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.135799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fc07d-0483-4574-9d73-904a703e8461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.136171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fc07d-0483-4574-9d73-904a703e8461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.137620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fc07d-0483-4574-9d73-904a703e8461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:22.139367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fc07d-0483-4574-9d73-904a703e8461 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:22.154248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1251/2518] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:22.158291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e68b6f-8768-4d38-9692-94d9123632fa admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:22.169701 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e68b6f-8768-4d38-9692-94d9123632fa admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:22.175655 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a90f18-65b1-4b77-9ef0-3c1a89f94dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:22.176839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a90f18-65b1-4b77-9ef0-3c1a89f94dac admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.177170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a90f18-65b1-4b77-9ef0-3c1a89f94dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.177391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a90f18-65b1-4b77-9ef0-3c1a89f94dac admin admin] PATH_INFO: `/v3/projects/9dc1d39a3d984984917948b1a3e17c92/users/63de2072968e4bf294a7d4797b1149b8/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.203439 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:22.205944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a90f18-65b1-4b77-9ef0-3c1a89f94dac admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dc1d39a3d984984917948b1a3e17c92, user_id=63de2072968e4bf294a7d4797b1149b8, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:22.208892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a90f18-65b1-4b77-9ef0-3c1a89f94dac admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:22.230061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1268/2519] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:22 2022] PUT /identity/v3/projects/9dc1d39a3d984984917948b1a3e17c92/users/63de2072968e4bf294a7d4797b1149b8/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:22.239917 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13339437-69e2-414d-8a08-c769558a2fc3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.240210 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13339437-69e2-414d-8a08-c769558a2fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.240481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13339437-69e2-414d-8a08-c769558a2fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.273389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-13339437-69e2-414d-8a08-c769558a2fc3 None None] MFA Rules not processed for user `63de2072968e4bf294a7d4797b1149b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:22.277249 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-13339437-69e2-414d-8a08-c769558a2fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:22.290588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:22.344474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a90f18-65b1-4b77-9ef0-3c1a89f94dac admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:22.347071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-04a90f18-65b1-4b77-9ef0-3c1a89f94dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:22.365829 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1252/2520] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:22.372889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13339437-69e2-414d-8a08-c769558a2fc3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:22.444615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a90f18-65b1-4b77-9ef0-3c1a89f94dac admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:22.449698 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:22.450965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.451243 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.451513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] PATH_INFO: `/v3/users/a20f6932d763461ea26c2d60ed58a59f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.452750 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20f6932d763461ea26c2d60ed58a59f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:22.454781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:22.456114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13339437-69e2-414d-8a08-c769558a2fc3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:22.460823 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f0f5-1d14-4b7e-ba48-771d8cee0a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:22.461506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f0f5-1d14-4b7e-ba48-771d8cee0a26 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.461814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f0f5-1d14-4b7e-ba48-771d8cee0a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.462089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f0f5-1d14-4b7e-ba48-771d8cee0a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.462769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f0f5-1d14-4b7e-ba48-771d8cee0a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:22.464657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f0f5-1d14-4b7e-ba48-771d8cee0a26 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:22.491183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1253/2521] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:22 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:22.495310 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f0f5-1d14-4b7e-ba48-771d8cee0a26 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:22.506500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20f6932d763461ea26c2d60ed58a59f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:22.514149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20f6932d763461ea26c2d60ed58a59f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:22.514864 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20f6932d763461ea26c2d60ed58a59f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:22.515906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20f6932d763461ea26c2d60ed58a59f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:22.529030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20f6932d763461ea26c2d60ed58a59f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:22.541653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1269/2522] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:22 2022] DELETE /identity/v3/users/a20f6932d763461ea26c2d60ed58a59f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:22.562579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f0f5-1d14-4b7e-ba48-771d8cee0a26 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:22.617400 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:22.631331 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab465bc8-04cd-41c8-bbed-d63e51256510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:22.631986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab465bc8-04cd-41c8-bbed-d63e51256510 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.632262 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab465bc8-04cd-41c8-bbed-d63e51256510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.632530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab465bc8-04cd-41c8-bbed-d63e51256510 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.633238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab465bc8-04cd-41c8-bbed-d63e51256510 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:22.635539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab465bc8-04cd-41c8-bbed-d63e51256510 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:22.660831 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1254/2523] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:22 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:24:22.662189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:22.663584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:22.677701 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce793f5c-39d4-4fba-9edd-db368e11eb16 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:22.682437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e586a-e8db-4fbd-94fb-45dcdfa5fa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:22.683098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e586a-e8db-4fbd-94fb-45dcdfa5fa71 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.683380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e586a-e8db-4fbd-94fb-45dcdfa5fa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.683653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e586a-e8db-4fbd-94fb-45dcdfa5fa71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.684978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e586a-e8db-4fbd-94fb-45dcdfa5fa71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:22.686777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e586a-e8db-4fbd-94fb-45dcdfa5fa71 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:22.697698 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1270/2524] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:22.702499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab465bc8-04cd-41c8-bbed-d63e51256510 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:22.717903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab465bc8-04cd-41c8-bbed-d63e51256510 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:22.724540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8d79f-8946-45c0-a348-fe71baf0fefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:22.725598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8d79f-8946-45c0-a348-fe71baf0fefa admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.725807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8d79f-8946-45c0-a348-fe71baf0fefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.726023 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8d79f-8946-45c0-a348-fe71baf0fefa admin admin] PATH_INFO: `/v3/projects/46c5f540286d444fa02ba765335dfc9a/users/b40de45273354386bc963f8f9c0e8328/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.734788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e586a-e8db-4fbd-94fb-45dcdfa5fa71 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:22.737039 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-699e586a-e8db-4fbd-94fb-45dcdfa5fa71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:22.745450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8d79f-8946-45c0-a348-fe71baf0fefa admin admin] RBAC: Authorizing `identity:create_grant(project_id=46c5f540286d444fa02ba765335dfc9a, user_id=b40de45273354386bc963f8f9c0e8328, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:22.748830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8d79f-8946-45c0-a348-fe71baf0fefa admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:22.752103 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e586a-e8db-4fbd-94fb-45dcdfa5fa71 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:22.758020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a5727-0344-425b-8870-62787ba51f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:22.759186 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a5727-0344-425b-8870-62787ba51f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.759814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a5727-0344-425b-8870-62787ba51f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.759935 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a5727-0344-425b-8870-62787ba51f20 admin admin] PATH_INFO: `/v3/projects/0c2a09a6e81240889c49b5abb24ab225` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.761056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a5727-0344-425b-8870-62787ba51f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c2a09a6e81240889c49b5abb24ab225)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:22.763289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a5727-0344-425b-8870-62787ba51f20 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:22.765839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1255/2525] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:22 2022] PUT /identity/v3/projects/46c5f540286d444fa02ba765335dfc9a/users/b40de45273354386bc963f8f9c0e8328/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:22.769621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8d79f-8946-45c0-a348-fe71baf0fefa admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:22.815022 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-572a5727-0344-425b-8870-62787ba51f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c2a09a6e81240889c49b5abb24ab225', 'request_id': 'req-572a5727-0344-425b-8870-62787ba51f20'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:22.821590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-572a5727-0344-425b-8870-62787ba51f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c2a09a6e81240889c49b5abb24ab225', 'request_id': 'req-572a5727-0344-425b-8870-62787ba51f20'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:22.822832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:22.829216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-572a5727-0344-425b-8870-62787ba51f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c2a09a6e81240889c49b5abb24ab225', 'request_id': 'req-572a5727-0344-425b-8870-62787ba51f20'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:22.832848 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8d79f-8946-45c0-a348-fe71baf0fefa admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:22.843053 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-572a5727-0344-425b-8870-62787ba51f20 admin admin] The token cache is being invalidate because project 0c2a09a6e81240889c49b5abb24ab225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:22.843371 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-572a5727-0344-425b-8870-62787ba51f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:22.844054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-572a5727-0344-425b-8870-62787ba51f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:22.845641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1271/2526] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:22 2022] DELETE /identity/v3/projects/0c2a09a6e81240889c49b5abb24ab225 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:22.850429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145d5f3-d670-46ec-a579-789c8cb3201b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.850772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145d5f3-d670-46ec-a579-789c8cb3201b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.851057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145d5f3-d670-46ec-a579-789c8cb3201b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.869511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1145d5f3-d670-46ec-a579-789c8cb3201b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:22.870667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1145d5f3-d670-46ec-a579-789c8cb3201b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:22.890805 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc78f6-4065-4d9d-8059-7d6b9b981ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:22.891720 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc78f6-4065-4d9d-8059-7d6b9b981ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.891890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc78f6-4065-4d9d-8059-7d6b9b981ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.892167 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc78f6-4065-4d9d-8059-7d6b9b981ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.892899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc78f6-4065-4d9d-8059-7d6b9b981ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:22.894536 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc78f6-4065-4d9d-8059-7d6b9b981ba2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:22.908371 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1256/2527] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:24:22.909429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc78f6-4065-4d9d-8059-7d6b9b981ba2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:22.911132 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-27cc78f6-4065-4d9d-8059-7d6b9b981ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:22.923979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc78f6-4065-4d9d-8059-7d6b9b981ba2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:22.928761 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1220b1a-4f98-4002-9bf8-feebea1e8627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:22.929612 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1220b1a-4f98-4002-9bf8-feebea1e8627 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.929919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1220b1a-4f98-4002-9bf8-feebea1e8627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.930188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1220b1a-4f98-4002-9bf8-feebea1e8627 admin admin] PATH_INFO: `/v3/projects/46c5f540286d444fa02ba765335dfc9a/users/b40de45273354386bc963f8f9c0e8328/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.932196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1220b1a-4f98-4002-9bf8-feebea1e8627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46c5f540286d444fa02ba765335dfc9a, user_id=b40de45273354386bc963f8f9c0e8328, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:22.932857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1272/2528] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:22.935412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1220b1a-4f98-4002-9bf8-feebea1e8627 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:22.940864 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e59bb-4ddf-44e5-8907-61e235f6581d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.941151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e59bb-4ddf-44e5-8907-61e235f6581d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.941423 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e59bb-4ddf-44e5-8907-61e235f6581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.945839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1257/2529] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:22 2022] PUT /identity/v3/projects/46c5f540286d444fa02ba765335dfc9a/users/b40de45273354386bc963f8f9c0e8328/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:22.957187 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c04a65-7b70-4757-8f37-2513ff0f9216 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:22.957471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c04a65-7b70-4757-8f37-2513ff0f9216 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:22.957774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c04a65-7b70-4757-8f37-2513ff0f9216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:22.959517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-059e59bb-4ddf-44e5-8907-61e235f6581d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:22.960710 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-059e59bb-4ddf-44e5-8907-61e235f6581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:22.985041 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b8c04a65-7b70-4757-8f37-2513ff0f9216 None None] MFA Rules not processed for user `b40de45273354386bc963f8f9c0e8328`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:22.986275 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b8c04a65-7b70-4757-8f37-2513ff0f9216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:23.019185 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:23.022193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1273/2530] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:23.026345 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e59bb-4ddf-44e5-8907-61e235f6581d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:23.036076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e59bb-4ddf-44e5-8907-61e235f6581d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:23.040673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beb5e7-ded7-4539-a148-384a36033900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:23.041309 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beb5e7-ded7-4539-a148-384a36033900 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:23.041584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beb5e7-ded7-4539-a148-384a36033900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:23.041887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beb5e7-ded7-4539-a148-384a36033900 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:23.042620 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beb5e7-ded7-4539-a148-384a36033900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:23.042918 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beb5e7-ded7-4539-a148-384a36033900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:23.045485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beb5e7-ded7-4539-a148-384a36033900 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:23.057273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1274/2531] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:24:23.058405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1258/2532] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:23.061577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beb5e7-ded7-4539-a148-384a36033900 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:23.069632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c04a65-7b70-4757-8f37-2513ff0f9216 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:23.071069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b8c04a65-7b70-4757-8f37-2513ff0f9216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:23.071183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beb5e7-ded7-4539-a148-384a36033900 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:23.076741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129ce21-6913-4738-a6d3-341460fde23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:23.077246 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129ce21-6913-4738-a6d3-341460fde23b admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:23.077527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129ce21-6913-4738-a6d3-341460fde23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:23.077837 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129ce21-6913-4738-a6d3-341460fde23b admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:23.078524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129ce21-6913-4738-a6d3-341460fde23b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:23.080439 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129ce21-6913-4738-a6d3-341460fde23b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:23.108101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1275/2533] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:23 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:23.112141 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129ce21-6913-4738-a6d3-341460fde23b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:23.131684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c04a65-7b70-4757-8f37-2513ff0f9216 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:23.164537 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:23.179279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129ce21-6913-4738-a6d3-341460fde23b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:23.184748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41ef4f-4671-431a-889d-4447a8585bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:23.185568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41ef4f-4671-431a-889d-4447a8585bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:23.185920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41ef4f-4671-431a-889d-4447a8585bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:23.186196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41ef4f-4671-431a-889d-4447a8585bd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:23.186888 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41ef4f-4671-431a-889d-4447a8585bd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:23.187877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5dd5b-9641-4f80-85b1-9c35e897f613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:23.188524 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5dd5b-9641-4f80-85b1-9c35e897f613 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:23.188807 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5dd5b-9641-4f80-85b1-9c35e897f613 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:23.188911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41ef4f-4671-431a-889d-4447a8585bd5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:23.189082 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5dd5b-9641-4f80-85b1-9c35e897f613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:23.212570 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1276/2534] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:23 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:24:23.213612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41ef4f-4671-431a-889d-4447a8585bd5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:23.223967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41ef4f-4671-431a-889d-4447a8585bd5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:23.228815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28830b4-a5e2-4003-a67b-7592e0e022e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:23.229454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28830b4-a5e2-4003-a67b-7592e0e022e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:23.229784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28830b4-a5e2-4003-a67b-7592e0e022e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:23.230053 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28830b4-a5e2-4003-a67b-7592e0e022e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:23.231006 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28830b4-a5e2-4003-a67b-7592e0e022e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:23.232669 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28830b4-a5e2-4003-a67b-7592e0e022e6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:23.242870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1277/2535] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:23.246728 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28830b4-a5e2-4003-a67b-7592e0e022e6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:23.251750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5dd5b-9641-4f80-85b1-9c35e897f613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:23.253198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5dd5b-9641-4f80-85b1-9c35e897f613 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:23.256377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28830b4-a5e2-4003-a67b-7592e0e022e6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:23.260980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b1fb0-a44a-4df5-ab5a-e7253e246f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:23.261617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b1fb0-a44a-4df5-ab5a-e7253e246f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:23.262515 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b1fb0-a44a-4df5-ab5a-e7253e246f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:23.262960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b1fb0-a44a-4df5-ab5a-e7253e246f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:23.263250 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1259/2536] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:24:23.263748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b1fb0-a44a-4df5-ab5a-e7253e246f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:23.265419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b1fb0-a44a-4df5-ab5a-e7253e246f99 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:23.291145 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1278/2537] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:24:23.292713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5dd5b-9641-4f80-85b1-9c35e897f613 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:23.304075 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5dd5b-9641-4f80-85b1-9c35e897f613 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:23.308383 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8e257-9caa-43d9-830b-93fb7ec14294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:23.309309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8e257-9caa-43d9-830b-93fb7ec14294 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:23.309668 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8e257-9caa-43d9-830b-93fb7ec14294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:23.309993 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8e257-9caa-43d9-830b-93fb7ec14294 admin admin] PATH_INFO: `/v3/projects/5abd1a9fb52f4ac4aab460faccd3ea13/users/f5c71a6ae5eb49ce86c4a2513686fce1/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:23.326483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8e257-9caa-43d9-830b-93fb7ec14294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5abd1a9fb52f4ac4aab460faccd3ea13, user_id=f5c71a6ae5eb49ce86c4a2513686fce1, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:23.329145 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8e257-9caa-43d9-830b-93fb7ec14294 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:23.344209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1260/2538] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:23 2022] PUT /identity/v3/projects/5abd1a9fb52f4ac4aab460faccd3ea13/users/f5c71a6ae5eb49ce86c4a2513686fce1/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:23.344405 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b1fb0-a44a-4df5-ab5a-e7253e246f99 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:23.354540 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b1fb0-a44a-4df5-ab5a-e7253e246f99 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:23.354772 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f4770-ca08-468a-ab1b-c544014ce26c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:23.355061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f4770-ca08-468a-ab1b-c544014ce26c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:23.355341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f4770-ca08-468a-ab1b-c544014ce26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:23.375056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:23.385542 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e94f4770-ca08-468a-ab1b-c544014ce26c None None] MFA Rules not processed for user `f5c71a6ae5eb49ce86c4a2513686fce1`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:23.387447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e94f4770-ca08-468a-ab1b-c544014ce26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:23.408625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a784dfd-ff6d-419e-9264-952504e732b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:23.409276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a784dfd-ff6d-419e-9264-952504e732b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:23.409559 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a784dfd-ff6d-419e-9264-952504e732b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:23.409872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a784dfd-ff6d-419e-9264-952504e732b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:23.411712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3a784dfd-ff6d-419e-9264-952504e732b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:23.454016 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1261/2539] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:23.465965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f4770-ca08-468a-ab1b-c544014ce26c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:23.467406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e94f4770-ca08-468a-ab1b-c544014ce26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:23.474170 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a784dfd-ff6d-419e-9264-952504e732b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:23.475790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a784dfd-ff6d-419e-9264-952504e732b0 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:23.485625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1279/2540] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:24:23.545332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f4770-ca08-468a-ab1b-c544014ce26c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:23.552097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89a993-832d-4973-bbf5-6c2cb3b068c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:23.552745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89a993-832d-4973-bbf5-6c2cb3b068c8 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:23.553241 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89a993-832d-4973-bbf5-6c2cb3b068c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:23.553524 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89a993-832d-4973-bbf5-6c2cb3b068c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:23.562793 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89a993-832d-4973-bbf5-6c2cb3b068c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:23.564788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89a993-832d-4973-bbf5-6c2cb3b068c8 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:23.574680 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1262/2541] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:24:26.249388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a784dfd-ff6d-419e-9264-952504e732b0 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:26.313761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a784dfd-ff6d-419e-9264-952504e732b0 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:26.318279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:26.319149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:26.319435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:26.319713 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] PATH_INFO: `/v3/users/f5c71a6ae5eb49ce86c4a2513686fce1` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:26.320902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c71a6ae5eb49ce86c4a2513686fce1)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:26.322886 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:26.336145 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:26.368600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c71a6ae5eb49ce86c4a2513686fce1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:26.375804 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c71a6ae5eb49ce86c4a2513686fce1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:26.376664 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c71a6ae5eb49ce86c4a2513686fce1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:26.377297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c71a6ae5eb49ce86c4a2513686fce1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:26.389595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c71a6ae5eb49ce86c4a2513686fce1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:26.402057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1280/2542] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:26 2022] DELETE /identity/v3/users/f5c71a6ae5eb49ce86c4a2513686fce1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:26.407223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad8301-5d02-4ac8-b257-44c1a69d8b1f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:26.407700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad8301-5d02-4ac8-b257-44c1a69d8b1f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:26.407795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad8301-5d02-4ac8-b257-44c1a69d8b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:26.424997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-07ad8301-5d02-4ac8-b257-44c1a69d8b1f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:26.426206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-07ad8301-5d02-4ac8-b257-44c1a69d8b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:26.487911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1263/2543] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:26.497853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:26.499703 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:26.556850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19569aff-0a13-419c-84b1-6f1e6df49a10 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:26.560661 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bf2bb-e19e-4f66-9b03-a6cede8c4b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:26.561298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bf2bb-e19e-4f66-9b03-a6cede8c4b12 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:26.561579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bf2bb-e19e-4f66-9b03-a6cede8c4b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:26.561951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bf2bb-e19e-4f66-9b03-a6cede8c4b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:26.571089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bf2bb-e19e-4f66-9b03-a6cede8c4b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:26.572461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bf2bb-e19e-4f66-9b03-a6cede8c4b12 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:26.581820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1281/2544] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:26.842750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad8301-5d02-4ac8-b257-44c1a69d8b1f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:26.844176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-07ad8301-5d02-4ac8-b257-44c1a69d8b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:26.857903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad8301-5d02-4ac8-b257-44c1a69d8b1f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:26.862578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c895d1c-5499-4b92-9f47-e061627dff6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:26.863431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c895d1c-5499-4b92-9f47-e061627dff6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:26.863733 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c895d1c-5499-4b92-9f47-e061627dff6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:26.864007 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c895d1c-5499-4b92-9f47-e061627dff6c admin admin] PATH_INFO: `/v3/projects/5abd1a9fb52f4ac4aab460faccd3ea13` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:26.865278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c895d1c-5499-4b92-9f47-e061627dff6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5abd1a9fb52f4ac4aab460faccd3ea13)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:26.867220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c895d1c-5499-4b92-9f47-e061627dff6c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:26.890321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:26.913343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3c895d1c-5499-4b92-9f47-e061627dff6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5abd1a9fb52f4ac4aab460faccd3ea13', 'request_id': 'req-3c895d1c-5499-4b92-9f47-e061627dff6c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:26.920755 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3c895d1c-5499-4b92-9f47-e061627dff6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5abd1a9fb52f4ac4aab460faccd3ea13', 'request_id': 'req-3c895d1c-5499-4b92-9f47-e061627dff6c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:26.927404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3c895d1c-5499-4b92-9f47-e061627dff6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5abd1a9fb52f4ac4aab460faccd3ea13', 'request_id': 'req-3c895d1c-5499-4b92-9f47-e061627dff6c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:26.937453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3c895d1c-5499-4b92-9f47-e061627dff6c admin admin] The token cache is being invalidate because project 5abd1a9fb52f4ac4aab460faccd3ea13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:26.937768 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3c895d1c-5499-4b92-9f47-e061627dff6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:26.938378 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3c895d1c-5499-4b92-9f47-e061627dff6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:26.939927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1264/2545] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:26 2022] DELETE /identity/v3/projects/5abd1a9fb52f4ac4aab460faccd3ea13 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:26.944857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd338677-67d9-4ee9-98af-2a9416c24d95 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:26.945142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd338677-67d9-4ee9-98af-2a9416c24d95 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:26.945414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd338677-67d9-4ee9-98af-2a9416c24d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:26.963139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-cd338677-67d9-4ee9-98af-2a9416c24d95 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:26.964323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cd338677-67d9-4ee9-98af-2a9416c24d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:27.024485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1282/2546] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:27.033035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a0ff1-eaed-4a6c-a656-e1e519b19b15 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:27.033325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a0ff1-eaed-4a6c-a656-e1e519b19b15 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:27.033600 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a0ff1-eaed-4a6c-a656-e1e519b19b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:27.051544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4a7a0ff1-eaed-4a6c-a656-e1e519b19b15 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:27.052979 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4a7a0ff1-eaed-4a6c-a656-e1e519b19b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:27.061019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1265/2547] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:27.065030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd338677-67d9-4ee9-98af-2a9416c24d95 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:27.075110 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd338677-67d9-4ee9-98af-2a9416c24d95 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:27.079828 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c1e21-aa3c-4cb6-beca-7e4831c130f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:27.080468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c1e21-aa3c-4cb6-beca-7e4831c130f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:27.080743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c1e21-aa3c-4cb6-beca-7e4831c130f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:27.081013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c1e21-aa3c-4cb6-beca-7e4831c130f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:27.081745 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c1e21-aa3c-4cb6-beca-7e4831c130f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:27.082056 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c1e21-aa3c-4cb6-beca-7e4831c130f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:27.083655 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c1e21-aa3c-4cb6-beca-7e4831c130f3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:27.095643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1283/2548] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:24:27.099760 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a0ff1-eaed-4a6c-a656-e1e519b19b15 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:27.109566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a0ff1-eaed-4a6c-a656-e1e519b19b15 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:27.114304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2dcb7e-2127-4cb8-9fa5-d6226e0993d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:27.114974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dcb7e-2127-4cb8-9fa5-d6226e0993d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:27.115254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dcb7e-2127-4cb8-9fa5-d6226e0993d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:27.115528 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dcb7e-2127-4cb8-9fa5-d6226e0993d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:27.116219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2dcb7e-2127-4cb8-9fa5-d6226e0993d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:27.118500 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2dcb7e-2127-4cb8-9fa5-d6226e0993d9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:27.144610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1266/2549] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:27 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:27.148982 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c1e21-aa3c-4cb6-beca-7e4831c130f3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:27.159947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c1e21-aa3c-4cb6-beca-7e4831c130f3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:27.178167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:27.219636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dcc89-695b-4a5e-a8c7-c0455d82ced6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:27.220333 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dcc89-695b-4a5e-a8c7-c0455d82ced6 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:27.220617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dcc89-695b-4a5e-a8c7-c0455d82ced6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:27.220893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dcc89-695b-4a5e-a8c7-c0455d82ced6 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:27.221571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dcc89-695b-4a5e-a8c7-c0455d82ced6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:27.223548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dcc89-695b-4a5e-a8c7-c0455d82ced6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:27.242944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1284/2550] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:27 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:27.247843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2dcb7e-2127-4cb8-9fa5-d6226e0993d9 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:27.257977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2dcb7e-2127-4cb8-9fa5-d6226e0993d9 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:27.263772 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebbc24-d267-4b8b-bd79-b62aaab041d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:27.264430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebbc24-d267-4b8b-bd79-b62aaab041d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:27.264706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebbc24-d267-4b8b-bd79-b62aaab041d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:27.264977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebbc24-d267-4b8b-bd79-b62aaab041d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:27.265691 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebbc24-d267-4b8b-bd79-b62aaab041d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:27.267396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebbc24-d267-4b8b-bd79-b62aaab041d8 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:27.278280 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1267/2551] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:27.282335 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dcc89-695b-4a5e-a8c7-c0455d82ced6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:27.292119 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dcc89-695b-4a5e-a8c7-c0455d82ced6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:27.297192 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec58388-f711-4809-a5a5-eb1a7f362cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:27.297869 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec58388-f711-4809-a5a5-eb1a7f362cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:27.298146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec58388-f711-4809-a5a5-eb1a7f362cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:27.298420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec58388-f711-4809-a5a5-eb1a7f362cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:27.299128 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec58388-f711-4809-a5a5-eb1a7f362cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:27.300718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec58388-f711-4809-a5a5-eb1a7f362cd7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:27.311626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1285/2552] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:27.316185 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebbc24-d267-4b8b-bd79-b62aaab041d8 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:27.327941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebbc24-d267-4b8b-bd79-b62aaab041d8 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:27.332522 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cafb0b-d90d-4d10-8aae-6c9e19ed24d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:27.333387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cafb0b-d90d-4d10-8aae-6c9e19ed24d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:27.333662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cafb0b-d90d-4d10-8aae-6c9e19ed24d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:27.333962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cafb0b-d90d-4d10-8aae-6c9e19ed24d0 admin admin] PATH_INFO: `/v3/projects/c7e89a6394714440939a97a168f8105a/users/88f278fcce4348fc8d8ddf4a5fb28ef0/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:27.348401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cafb0b-d90d-4d10-8aae-6c9e19ed24d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7e89a6394714440939a97a168f8105a, user_id=88f278fcce4348fc8d8ddf4a5fb28ef0, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:27.351000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cafb0b-d90d-4d10-8aae-6c9e19ed24d0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:27.363925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1268/2553] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:27 2022] PUT /identity/v3/projects/c7e89a6394714440939a97a168f8105a/users/88f278fcce4348fc8d8ddf4a5fb28ef0/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:27.375164 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb1738-695b-4b14-b40e-9b231e047888 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:27.375524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb1738-695b-4b14-b40e-9b231e047888 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:27.375865 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb1738-695b-4b14-b40e-9b231e047888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:27.403271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-91fb1738-695b-4b14-b40e-9b231e047888 None None] MFA Rules not processed for user `88f278fcce4348fc8d8ddf4a5fb28ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:27.404506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-91fb1738-695b-4b14-b40e-9b231e047888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:27.469004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1286/2554] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:27.479800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cafb0b-d90d-4d10-8aae-6c9e19ed24d0 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:27.481583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-33cafb0b-d90d-4d10-8aae-6c9e19ed24d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:27.538975 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cafb0b-d90d-4d10-8aae-6c9e19ed24d0 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:27.542639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd130a-5c4b-474a-849b-595858d5b224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:27.543856 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd130a-5c4b-474a-849b-595858d5b224 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:27.544229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd130a-5c4b-474a-849b-595858d5b224 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:27.544506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd130a-5c4b-474a-849b-595858d5b224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:27.553209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd130a-5c4b-474a-849b-595858d5b224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:27.554669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd130a-5c4b-474a-849b-595858d5b224 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:27.564850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1269/2555] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:32.272727 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb1738-695b-4b14-b40e-9b231e047888 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:32.274130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-91fb1738-695b-4b14-b40e-9b231e047888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:32.344196 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb1738-695b-4b14-b40e-9b231e047888 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:32.348685 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:32.349576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:32.349898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:32.350172 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] PATH_INFO: `/v3/users/63de2072968e4bf294a7d4797b1149b8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:32.351354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63de2072968e4bf294a7d4797b1149b8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:32.353284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:32.387120 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:32.410116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63de2072968e4bf294a7d4797b1149b8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:32.417648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63de2072968e4bf294a7d4797b1149b8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:32.418331 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63de2072968e4bf294a7d4797b1149b8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:32.418954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63de2072968e4bf294a7d4797b1149b8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:32.431265 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63de2072968e4bf294a7d4797b1149b8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:32.444076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1287/2556] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:32 2022] DELETE /identity/v3/users/63de2072968e4bf294a7d4797b1149b8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:32.448586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8efa63-8a05-4afa-aba1-196f2ae900bd None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:32.449061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8efa63-8a05-4afa-aba1-196f2ae900bd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:32.449159 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8efa63-8a05-4afa-aba1-196f2ae900bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:32.467033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ca8efa63-8a05-4afa-aba1-196f2ae900bd None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:32.468278 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ca8efa63-8a05-4afa-aba1-196f2ae900bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:32.511850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:32.528802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1270/2557] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:32.538463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:32.539904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:32.597414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da39fbb-1dc6-47e1-8f55-7647fda307b7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:32.601088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9f2ad-2801-460b-ac2f-a2bc9e872ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:32.601744 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9f2ad-2801-460b-ac2f-a2bc9e872ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:32.602042 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9f2ad-2801-460b-ac2f-a2bc9e872ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:32.602315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9f2ad-2801-460b-ac2f-a2bc9e872ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:32.611422 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9f2ad-2801-460b-ac2f-a2bc9e872ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:32.612868 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9f2ad-2801-460b-ac2f-a2bc9e872ee4 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:32.622183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1288/2558] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:32.914618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8efa63-8a05-4afa-aba1-196f2ae900bd None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:32.916334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ca8efa63-8a05-4afa-aba1-196f2ae900bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:32.930176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8efa63-8a05-4afa-aba1-196f2ae900bd None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:32.936618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:32.937644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:32.938070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:32.938254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] PATH_INFO: `/v3/projects/9dc1d39a3d984984917948b1a3e17c92` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:32.939443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dc1d39a3d984984917948b1a3e17c92)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:32.941348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:32.991283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dc1d39a3d984984917948b1a3e17c92', 'request_id': 'req-e295613a-88f1-427e-b496-9daa5b0164bd'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:32.999489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dc1d39a3d984984917948b1a3e17c92', 'request_id': 'req-e295613a-88f1-427e-b496-9daa5b0164bd'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.005799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dc1d39a3d984984917948b1a3e17c92', 'request_id': 'req-e295613a-88f1-427e-b496-9daa5b0164bd'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.016756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] The token cache is being invalidate because project 9dc1d39a3d984984917948b1a3e17c92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:33.017042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.017681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.019243 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1271/2559] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:32 2022] DELETE /identity/v3/projects/9dc1d39a3d984984917948b1a3e17c92 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:33.023289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9f2ad-2801-460b-ac2f-a2bc9e872ee4 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:33.034075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9f2ad-2801-460b-ac2f-a2bc9e872ee4 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:33.095093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae1c89-702d-43d4-a667-62e950648bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:33.095974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae1c89-702d-43d4-a667-62e950648bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:33.096254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae1c89-702d-43d4-a667-62e950648bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:33.096526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae1c89-702d-43d4-a667-62e950648bc8 admin admin] PATH_INFO: `/v3/users/b40de45273354386bc963f8f9c0e8328` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:33.098340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae1c89-702d-43d4-a667-62e950648bc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b40de45273354386bc963f8f9c0e8328)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:33.102082 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae1c89-702d-43d4-a667-62e950648bc8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:33.170696 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0dae1c89-702d-43d4-a667-62e950648bc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b40de45273354386bc963f8f9c0e8328'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.175161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:33.180376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0dae1c89-702d-43d4-a667-62e950648bc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b40de45273354386bc963f8f9c0e8328'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.181565 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0dae1c89-702d-43d4-a667-62e950648bc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b40de45273354386bc963f8f9c0e8328'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.182429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0dae1c89-702d-43d4-a667-62e950648bc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b40de45273354386bc963f8f9c0e8328'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.195724 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-0dae1c89-702d-43d4-a667-62e950648bc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b40de45273354386bc963f8f9c0e8328'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.208253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1289/2560] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:33 2022] DELETE /identity/v3/users/b40de45273354386bc963f8f9c0e8328 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:33.502220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:33.504623 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:33.534212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295613a-88f1-427e-b496-9daa5b0164bd admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:33.597008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:33.597908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:33.598275 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:33.598549 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9 admin admin] PATH_INFO: `/v3/projects/46c5f540286d444fa02ba765335dfc9a` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:33.599776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46c5f540286d444fa02ba765335dfc9a)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:33.602105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:33.660171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46c5f540286d444fa02ba765335dfc9a', 'request_id': 'req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.667889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46c5f540286d444fa02ba765335dfc9a', 'request_id': 'req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.674404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46c5f540286d444fa02ba765335dfc9a', 'request_id': 'req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.684746 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9 admin admin] The token cache is being invalidate because project 46c5f540286d444fa02ba765335dfc9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:33.685112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.685961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ebc806e4-84e1-4b4d-90c2-2b3878b205a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:33.687492 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1272/2561] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:33 2022] DELETE /identity/v3/projects/46c5f540286d444fa02ba765335dfc9a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:33.692377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63f3ee-ab0d-4084-bc43-07792c93287b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:33.692937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63f3ee-ab0d-4084-bc43-07792c93287b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:33.692937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63f3ee-ab0d-4084-bc43-07792c93287b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:33.711776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6f63f3ee-ab0d-4084-bc43-07792c93287b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:33.713085 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6f63f3ee-ab0d-4084-bc43-07792c93287b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:33.777496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1290/2562] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:33.786650 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f89a-b1cf-467b-bdfe-f5e785782f16 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:33.786936 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f89a-b1cf-467b-bdfe-f5e785782f16 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:33.787288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f89a-b1cf-467b-bdfe-f5e785782f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:33.794149 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:33.807819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4a29f89a-b1cf-467b-bdfe-f5e785782f16 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:33.809630 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4a29f89a-b1cf-467b-bdfe-f5e785782f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:33.820008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1273/2563] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:33.823879 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63f3ee-ab0d-4084-bc43-07792c93287b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:33.834194 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63f3ee-ab0d-4084-bc43-07792c93287b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:33.839185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef100c87-d778-4bf5-a2b4-b1c67baf5e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:33.839965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef100c87-d778-4bf5-a2b4-b1c67baf5e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:33.840245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef100c87-d778-4bf5-a2b4-b1c67baf5e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:33.840523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef100c87-d778-4bf5-a2b4-b1c67baf5e3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:33.842061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef100c87-d778-4bf5-a2b4-b1c67baf5e3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:33.842441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef100c87-d778-4bf5-a2b4-b1c67baf5e3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:33.844276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef100c87-d778-4bf5-a2b4-b1c67baf5e3e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:33.855974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1291/2564] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:24:33.860296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29f89a-b1cf-467b-bdfe-f5e785782f16 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:33.869517 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29f89a-b1cf-467b-bdfe-f5e785782f16 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:33.874506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43203404-5e30-4100-b44e-aa0b5fcfb55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:33.875156 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43203404-5e30-4100-b44e-aa0b5fcfb55b admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:33.875439 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43203404-5e30-4100-b44e-aa0b5fcfb55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:33.875714 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43203404-5e30-4100-b44e-aa0b5fcfb55b admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:33.876404 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43203404-5e30-4100-b44e-aa0b5fcfb55b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:33.878338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43203404-5e30-4100-b44e-aa0b5fcfb55b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:33.910302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1274/2565] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:33 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:33.915326 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef100c87-d778-4bf5-a2b4-b1c67baf5e3e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:33.925015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef100c87-d778-4bf5-a2b4-b1c67baf5e3e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:33.980296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bbda3-e32e-4372-8bd6-f08578783471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:33.980951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bbda3-e32e-4372-8bd6-f08578783471 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:33.981231 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bbda3-e32e-4372-8bd6-f08578783471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:33.981503 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bbda3-e32e-4372-8bd6-f08578783471 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:33.982392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bbda3-e32e-4372-8bd6-f08578783471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:33.984319 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bbda3-e32e-4372-8bd6-f08578783471 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:34.004270 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1292/2566] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:33 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:34.008662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43203404-5e30-4100-b44e-aa0b5fcfb55b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:34.019645 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43203404-5e30-4100-b44e-aa0b5fcfb55b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:34.024403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8da89-fb42-491f-86e8-8cfefea40352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:34.025090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8da89-fb42-491f-86e8-8cfefea40352 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:34.025370 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8da89-fb42-491f-86e8-8cfefea40352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:34.025776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8da89-fb42-491f-86e8-8cfefea40352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:34.026440 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8da89-fb42-491f-86e8-8cfefea40352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:34.028030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8da89-fb42-491f-86e8-8cfefea40352 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:34.039504 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1275/2567] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:24:34.043287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bbda3-e32e-4372-8bd6-f08578783471 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:34.053632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bbda3-e32e-4372-8bd6-f08578783471 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:34.058202 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e37a2-b955-476d-9c2a-f0a94dd021f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:34.058851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e37a2-b955-476d-9c2a-f0a94dd021f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:34.059136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e37a2-b955-476d-9c2a-f0a94dd021f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:34.059414 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e37a2-b955-476d-9c2a-f0a94dd021f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:34.060625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e37a2-b955-476d-9c2a-f0a94dd021f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:34.062276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e37a2-b955-476d-9c2a-f0a94dd021f0 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:34.073692 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1293/2568] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:34.079000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8da89-fb42-491f-86e8-8cfefea40352 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:34.090418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8da89-fb42-491f-86e8-8cfefea40352 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:34.095272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688071ee-d0b4-4c42-929a-6d10439bdbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:34.096144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688071ee-d0b4-4c42-929a-6d10439bdbd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:34.096423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688071ee-d0b4-4c42-929a-6d10439bdbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:34.096696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688071ee-d0b4-4c42-929a-6d10439bdbd4 admin admin] PATH_INFO: `/v3/projects/c2ee0608d1014b89845e58ce8ff0f65d/users/58181c6a32e14d0ea48d07983ffdc85e/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:34.111344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688071ee-d0b4-4c42-929a-6d10439bdbd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ee0608d1014b89845e58ce8ff0f65d, user_id=58181c6a32e14d0ea48d07983ffdc85e, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:34.114117 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688071ee-d0b4-4c42-929a-6d10439bdbd4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:34.128382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1276/2569] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:34 2022] PUT /identity/v3/projects/c2ee0608d1014b89845e58ce8ff0f65d/users/58181c6a32e14d0ea48d07983ffdc85e/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:34.140401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198b89b-d102-4ad1-9aa0-02e55a80b7c6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:34.140689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198b89b-d102-4ad1-9aa0-02e55a80b7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:34.140968 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198b89b-d102-4ad1-9aa0-02e55a80b7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:34.171209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d198b89b-d102-4ad1-9aa0-02e55a80b7c6 None None] MFA Rules not processed for user `58181c6a32e14d0ea48d07983ffdc85e`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:34.172812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d198b89b-d102-4ad1-9aa0-02e55a80b7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:34.205319 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:34.264037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1294/2570] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:34.269115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688071ee-d0b4-4c42-929a-6d10439bdbd4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:34.349091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688071ee-d0b4-4c42-929a-6d10439bdbd4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:34.353886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e75453-1bab-4745-bb85-2ef8a7446a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:34.354683 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e75453-1bab-4745-bb85-2ef8a7446a90 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:34.355060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e75453-1bab-4745-bb85-2ef8a7446a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:34.355336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e75453-1bab-4745-bb85-2ef8a7446a90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:34.356114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e75453-1bab-4745-bb85-2ef8a7446a90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:34.360105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e75453-1bab-4745-bb85-2ef8a7446a90 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:34.390470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1277/2571] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:34 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:34.395281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198b89b-d102-4ad1-9aa0-02e55a80b7c6 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:34.410059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198b89b-d102-4ad1-9aa0-02e55a80b7c6 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:34.483505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983795d-3768-4de7-a181-db4aa4a6714a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:34.484187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983795d-3768-4de7-a181-db4aa4a6714a admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:34.484465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983795d-3768-4de7-a181-db4aa4a6714a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:34.484737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983795d-3768-4de7-a181-db4aa4a6714a admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:34.485427 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983795d-3768-4de7-a181-db4aa4a6714a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:34.487432 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983795d-3768-4de7-a181-db4aa4a6714a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:34.514094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1295/2572] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:34 2022] POST /identity/v3/users => generated 300 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:34.518059 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e75453-1bab-4745-bb85-2ef8a7446a90 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:34.529962 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e75453-1bab-4745-bb85-2ef8a7446a90 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:34.534772 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3c99e-6868-4a85-9beb-90c778f48d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:34.535449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3c99e-6868-4a85-9beb-90c778f48d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:34.535725 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3c99e-6868-4a85-9beb-90c778f48d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:34.536056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3c99e-6868-4a85-9beb-90c778f48d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:34.536794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3c99e-6868-4a85-9beb-90c778f48d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:34.538428 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3c99e-6868-4a85-9beb-90c778f48d98 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:34.550689 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1278/2573] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:34.555482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983795d-3768-4de7-a181-db4aa4a6714a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:34.566646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983795d-3768-4de7-a181-db4aa4a6714a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:34.571387 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bea833-d801-4fe6-bb06-a3780201ec70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:34.572276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea833-d801-4fe6-bb06-a3780201ec70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:34.572612 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea833-d801-4fe6-bb06-a3780201ec70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:34.572830 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea833-d801-4fe6-bb06-a3780201ec70 admin admin] PATH_INFO: `/v3/projects/00a8b23ab2c54f2c8ae67fc34f1a901b/users/e058da30516344fe9bc599f55f9867ea/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:34.589159 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bea833-d801-4fe6-bb06-a3780201ec70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a8b23ab2c54f2c8ae67fc34f1a901b, user_id=e058da30516344fe9bc599f55f9867ea, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:34.591927 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bea833-d801-4fe6-bb06-a3780201ec70 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:34.609087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1296/2574] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:34 2022] PUT /identity/v3/projects/00a8b23ab2c54f2c8ae67fc34f1a901b/users/e058da30516344fe9bc599f55f9867ea/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:34.614243 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3c99e-6868-4a85-9beb-90c778f48d98 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:34.629592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3c99e-6868-4a85-9beb-90c778f48d98 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:34.707895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b41a1-4920-4ef6-82d3-836c7213dac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:34.709031 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b41a1-4920-4ef6-82d3-836c7213dac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:34.709341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b41a1-4920-4ef6-82d3-836c7213dac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:34.709819 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b41a1-4920-4ef6-82d3-836c7213dac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:34.710606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b41a1-4920-4ef6-82d3-836c7213dac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:34.712206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b41a1-4920-4ef6-82d3-836c7213dac5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:34.723862 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1279/2575] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:24:34.728955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bea833-d801-4fe6-bb06-a3780201ec70 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:34.743699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bea833-d801-4fe6-bb06-a3780201ec70 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:34.749538 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a430ac-c30d-4672-915c-0e880875963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:34.750841 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a430ac-c30d-4672-915c-0e880875963f admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:34.751118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a430ac-c30d-4672-915c-0e880875963f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:34.751565 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a430ac-c30d-4672-915c-0e880875963f admin admin] PATH_INFO: `/v3/projects/00a8b23ab2c54f2c8ae67fc34f1a901b/users/e058da30516344fe9bc599f55f9867ea/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:34.753981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a430ac-c30d-4672-915c-0e880875963f admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a8b23ab2c54f2c8ae67fc34f1a901b, user_id=e058da30516344fe9bc599f55f9867ea, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:34.757979 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a430ac-c30d-4672-915c-0e880875963f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:34.774808 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1297/2576] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:34 2022] PUT /identity/v3/projects/00a8b23ab2c54f2c8ae67fc34f1a901b/users/e058da30516344fe9bc599f55f9867ea/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:34.790914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117503-59e9-41b7-a633-dec0ff6d2354 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:34.791832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117503-59e9-41b7-a633-dec0ff6d2354 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:34.791832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117503-59e9-41b7-a633-dec0ff6d2354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:34.813788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:34.825393 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-78117503-59e9-41b7-a633-dec0ff6d2354 None None] MFA Rules not processed for user `e058da30516344fe9bc599f55f9867ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:34.827391 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-78117503-59e9-41b7-a633-dec0ff6d2354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:34.917076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1280/2577] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:34.934697 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a430ac-c30d-4672-915c-0e880875963f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:34.936675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-95a430ac-c30d-4672-915c-0e880875963f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:35.006757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a430ac-c30d-4672-915c-0e880875963f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:35.011054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedd4aa-f29e-4ee3-b2fd-3be4f88fee71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:35.011708 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedd4aa-f29e-4ee3-b2fd-3be4f88fee71 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:35.012131 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedd4aa-f29e-4ee3-b2fd-3be4f88fee71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:35.012467 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedd4aa-f29e-4ee3-b2fd-3be4f88fee71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:35.023965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedd4aa-f29e-4ee3-b2fd-3be4f88fee71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:35.025468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedd4aa-f29e-4ee3-b2fd-3be4f88fee71 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:35.036514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1298/2578] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:35.106239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78117503-59e9-41b7-a633-dec0ff6d2354 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:35.170939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78117503-59e9-41b7-a633-dec0ff6d2354 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:35.175476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:35.176341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:35.176624 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:35.177154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] PATH_INFO: `/v3/users/58181c6a32e14d0ea48d07983ffdc85e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:35.178390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] RBAC: Authorizing `identity:delete_user(user_id=58181c6a32e14d0ea48d07983ffdc85e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:35.180308 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:35.225916 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58181c6a32e14d0ea48d07983ffdc85e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.233655 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58181c6a32e14d0ea48d07983ffdc85e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.245606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58181c6a32e14d0ea48d07983ffdc85e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.246266 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58181c6a32e14d0ea48d07983ffdc85e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.253332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58181c6a32e14d0ea48d07983ffdc85e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.259184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1281/2579] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:35 2022] DELETE /identity/v3/users/58181c6a32e14d0ea48d07983ffdc85e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:35.264507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63517e-a1ba-4041-997c-ea8b67588bc0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:35.264803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63517e-a1ba-4041-997c-ea8b67588bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:35.265109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63517e-a1ba-4041-997c-ea8b67588bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:35.283484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1f63517e-a1ba-4041-997c-ea8b67588bc0 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:35.284813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1f63517e-a1ba-4041-997c-ea8b67588bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:35.298281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:35.350045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1299/2580] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:35.359636 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:35.361044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:35.375496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:35.421471 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840c938-e639-46da-928a-aa1254a9a5af admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:35.425121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb622b4-e913-4248-90d7-f182be59705f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:35.425793 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb622b4-e913-4248-90d7-f182be59705f service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:35.426080 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb622b4-e913-4248-90d7-f182be59705f service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:35.426352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb622b4-e913-4248-90d7-f182be59705f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:35.436131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb622b4-e913-4248-90d7-f182be59705f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:35.437555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb622b4-e913-4248-90d7-f182be59705f service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:35.447199 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1282/2581] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:35.715096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63517e-a1ba-4041-997c-ea8b67588bc0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:35.716765 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1f63517e-a1ba-4041-997c-ea8b67588bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:35.728971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63517e-a1ba-4041-997c-ea8b67588bc0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:35.733858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:35.734719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:35.734999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:35.735273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] PATH_INFO: `/v3/projects/c2ee0608d1014b89845e58ce8ff0f65d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:35.736445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ee0608d1014b89845e58ce8ff0f65d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:35.738474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:35.786179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ee0608d1014b89845e58ce8ff0f65d', 'request_id': 'req-e0d428a7-fec4-416e-90ae-4719bccbad5a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.792227 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ee0608d1014b89845e58ce8ff0f65d', 'request_id': 'req-e0d428a7-fec4-416e-90ae-4719bccbad5a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.798320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ee0608d1014b89845e58ce8ff0f65d', 'request_id': 'req-e0d428a7-fec4-416e-90ae-4719bccbad5a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.811666 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] The token cache is being invalidate because project c2ee0608d1014b89845e58ce8ff0f65d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:35.811955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.812583 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.814152 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1300/2582] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:35 2022] DELETE /identity/v3/projects/c2ee0608d1014b89845e58ce8ff0f65d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:35.818288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb622b4-e913-4248-90d7-f182be59705f service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:35.828128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb622b4-e913-4248-90d7-f182be59705f service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:35.887055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187209dc-03a3-4b8e-aec0-7bc011f5c4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:35.888036 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187209dc-03a3-4b8e-aec0-7bc011f5c4de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:35.888269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187209dc-03a3-4b8e-aec0-7bc011f5c4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:35.888631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187209dc-03a3-4b8e-aec0-7bc011f5c4de admin admin] PATH_INFO: `/v3/users/e058da30516344fe9bc599f55f9867ea` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:35.889742 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187209dc-03a3-4b8e-aec0-7bc011f5c4de admin admin] RBAC: Authorizing `identity:delete_user(user_id=e058da30516344fe9bc599f55f9867ea)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:35.891652 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187209dc-03a3-4b8e-aec0-7bc011f5c4de admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:35.948359 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-187209dc-03a3-4b8e-aec0-7bc011f5c4de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e058da30516344fe9bc599f55f9867ea'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.956933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-187209dc-03a3-4b8e-aec0-7bc011f5c4de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e058da30516344fe9bc599f55f9867ea'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.969216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-187209dc-03a3-4b8e-aec0-7bc011f5c4de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e058da30516344fe9bc599f55f9867ea'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.969912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-187209dc-03a3-4b8e-aec0-7bc011f5c4de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e058da30516344fe9bc599f55f9867ea'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.976532 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-187209dc-03a3-4b8e-aec0-7bc011f5c4de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e058da30516344fe9bc599f55f9867ea'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:35.982773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1283/2583] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:35 2022] DELETE /identity/v3/users/e058da30516344fe9bc599f55f9867ea => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:36.256248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:36.257784 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:36.276819 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d428a7-fec4-416e-90ae-4719bccbad5a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:36.340069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be360619-52c9-4395-94cc-cf260a286f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:36.340918 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be360619-52c9-4395-94cc-cf260a286f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:36.341887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be360619-52c9-4395-94cc-cf260a286f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:36.342227 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be360619-52c9-4395-94cc-cf260a286f68 admin admin] PATH_INFO: `/v3/projects/00a8b23ab2c54f2c8ae67fc34f1a901b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:36.343402 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be360619-52c9-4395-94cc-cf260a286f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a8b23ab2c54f2c8ae67fc34f1a901b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:36.345407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be360619-52c9-4395-94cc-cf260a286f68 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:36.391054 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:36.394788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-be360619-52c9-4395-94cc-cf260a286f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a8b23ab2c54f2c8ae67fc34f1a901b', 'request_id': 'req-be360619-52c9-4395-94cc-cf260a286f68'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:36.402526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-be360619-52c9-4395-94cc-cf260a286f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a8b23ab2c54f2c8ae67fc34f1a901b', 'request_id': 'req-be360619-52c9-4395-94cc-cf260a286f68'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:36.408815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-be360619-52c9-4395-94cc-cf260a286f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a8b23ab2c54f2c8ae67fc34f1a901b', 'request_id': 'req-be360619-52c9-4395-94cc-cf260a286f68'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:36.422768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-be360619-52c9-4395-94cc-cf260a286f68 admin admin] The token cache is being invalidate because project 00a8b23ab2c54f2c8ae67fc34f1a901b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:36.423060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-be360619-52c9-4395-94cc-cf260a286f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:36.423684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-be360619-52c9-4395-94cc-cf260a286f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:36.425223 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1301/2584] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:36 2022] DELETE /identity/v3/projects/00a8b23ab2c54f2c8ae67fc34f1a901b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:36.430377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cdb78-3cb4-4eb5-b1f7-d7cf90ec3e7d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:36.430610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cdb78-3cb4-4eb5-b1f7-d7cf90ec3e7d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:36.430889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cdb78-3cb4-4eb5-b1f7-d7cf90ec3e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:36.449155 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2f9cdb78-3cb4-4eb5-b1f7-d7cf90ec3e7d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:36.451016 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2f9cdb78-3cb4-4eb5-b1f7-d7cf90ec3e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:36.481126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:36.512767 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1284/2585] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:36.520994 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0f8af-0749-41fb-b770-3b08e9a5e0a8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:36.521286 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0f8af-0749-41fb-b770-3b08e9a5e0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:36.521566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0f8af-0749-41fb-b770-3b08e9a5e0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:36.539688 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-62e0f8af-0749-41fb-b770-3b08e9a5e0a8 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:36.540876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-62e0f8af-0749-41fb-b770-3b08e9a5e0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:36.549118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1302/2586] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:36.553536 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cdb78-3cb4-4eb5-b1f7-d7cf90ec3e7d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:36.564188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cdb78-3cb4-4eb5-b1f7-d7cf90ec3e7d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:36.568758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bce0b3-208e-4812-ab57-8529537975d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:36.569405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bce0b3-208e-4812-ab57-8529537975d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:36.569688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bce0b3-208e-4812-ab57-8529537975d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:36.569992 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bce0b3-208e-4812-ab57-8529537975d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:36.570710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bce0b3-208e-4812-ab57-8529537975d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:36.571016 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bce0b3-208e-4812-ab57-8529537975d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:36.572748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bce0b3-208e-4812-ab57-8529537975d6 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:36.584555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1285/2587] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:24:36.588673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0f8af-0749-41fb-b770-3b08e9a5e0a8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:36.598044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0f8af-0749-41fb-b770-3b08e9a5e0a8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:36.602722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47e140-22ec-4ad5-bb2b-1f2dcdb0021e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:36.603366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47e140-22ec-4ad5-bb2b-1f2dcdb0021e admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:36.603648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47e140-22ec-4ad5-bb2b-1f2dcdb0021e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:36.603920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47e140-22ec-4ad5-bb2b-1f2dcdb0021e admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:36.604653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47e140-22ec-4ad5-bb2b-1f2dcdb0021e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:36.607280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47e140-22ec-4ad5-bb2b-1f2dcdb0021e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:36.632673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1303/2588] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:36 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:36.636760 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bce0b3-208e-4812-ab57-8529537975d6 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:36.647021 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bce0b3-208e-4812-ab57-8529537975d6 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:36.702982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa88528-a045-4247-954c-e4a3cf8acf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:36.703570 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa88528-a045-4247-954c-e4a3cf8acf14 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:36.703855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa88528-a045-4247-954c-e4a3cf8acf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:36.704126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa88528-a045-4247-954c-e4a3cf8acf14 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:36.704811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa88528-a045-4247-954c-e4a3cf8acf14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:36.706749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa88528-a045-4247-954c-e4a3cf8acf14 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:36.726420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1286/2589] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:36 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:24:36.730463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47e140-22ec-4ad5-bb2b-1f2dcdb0021e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:36.741509 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47e140-22ec-4ad5-bb2b-1f2dcdb0021e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:36.746133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853a5ec-5c27-4d5c-9989-3a7034e49f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:36.746788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853a5ec-5c27-4d5c-9989-3a7034e49f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:36.747066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853a5ec-5c27-4d5c-9989-3a7034e49f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:36.748052 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853a5ec-5c27-4d5c-9989-3a7034e49f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:36.748773 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853a5ec-5c27-4d5c-9989-3a7034e49f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:36.750444 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853a5ec-5c27-4d5c-9989-3a7034e49f6d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:36.760729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1304/2590] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:36.764749 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa88528-a045-4247-954c-e4a3cf8acf14 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:36.774997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa88528-a045-4247-954c-e4a3cf8acf14 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:36.779988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7def3cc-f708-48fc-b7aa-6d1cc6e65d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:36.780631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7def3cc-f708-48fc-b7aa-6d1cc6e65d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:36.780933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7def3cc-f708-48fc-b7aa-6d1cc6e65d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:36.781231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7def3cc-f708-48fc-b7aa-6d1cc6e65d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:36.781984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7def3cc-f708-48fc-b7aa-6d1cc6e65d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:36.783548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7def3cc-f708-48fc-b7aa-6d1cc6e65d38 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:36.793324 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1287/2591] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:36.797880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853a5ec-5c27-4d5c-9989-3a7034e49f6d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:36.807844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853a5ec-5c27-4d5c-9989-3a7034e49f6d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:36.813514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97154906-64ae-4376-bf75-b6d65ef32095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:36.814573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97154906-64ae-4376-bf75-b6d65ef32095 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:36.814852 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97154906-64ae-4376-bf75-b6d65ef32095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:36.815202 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97154906-64ae-4376-bf75-b6d65ef32095 admin admin] PATH_INFO: `/v3/projects/2ec551d1b501453a918e9603521cf59b/users/a8a51acb4036485380d2850b028aefcb/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:36.831859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97154906-64ae-4376-bf75-b6d65ef32095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec551d1b501453a918e9603521cf59b, user_id=a8a51acb4036485380d2850b028aefcb, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:36.834591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97154906-64ae-4376-bf75-b6d65ef32095 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:36.849281 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1305/2592] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:36 2022] PUT /identity/v3/projects/2ec551d1b501453a918e9603521cf59b/users/a8a51acb4036485380d2850b028aefcb/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:36.861351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c44657-a58d-4ef4-a8a6-137d5b640ca4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:36.861899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c44657-a58d-4ef4-a8a6-137d5b640ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:36.862033 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c44657-a58d-4ef4-a8a6-137d5b640ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:36.888703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-93c44657-a58d-4ef4-a8a6-137d5b640ca4 None None] MFA Rules not processed for user `a8a51acb4036485380d2850b028aefcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:36.889937 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-93c44657-a58d-4ef4-a8a6-137d5b640ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:36.958461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1288/2593] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:36.970446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97154906-64ae-4376-bf75-b6d65ef32095 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:36.972406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-97154906-64ae-4376-bf75-b6d65ef32095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:36.996651 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c44657-a58d-4ef4-a8a6-137d5b640ca4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:36.998447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-93c44657-a58d-4ef4-a8a6-137d5b640ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:37.038870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97154906-64ae-4376-bf75-b6d65ef32095 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:37.043519 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46d2df-39c2-4515-9dc4-6e7afcd9bc23 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a67120485f74f8bb0ff188cd3abd21c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:37.044045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:37.044849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46d2df-39c2-4515-9dc4-6e7afcd9bc23 service glance] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:37.045131 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46d2df-39c2-4515-9dc4-6e7afcd9bc23 service glance] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:37.045406 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46d2df-39c2-4515-9dc4-6e7afcd9bc23 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:37.056105 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46d2df-39c2-4515-9dc4-6e7afcd9bc23 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:37.057606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46d2df-39c2-4515-9dc4-6e7afcd9bc23 service glance] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:37.069454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1306/2594] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:37.079765 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c44657-a58d-4ef4-a8a6-137d5b640ca4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:37.084472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:37.085339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:37.085619 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:37.085921 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] PATH_INFO: `/v3/users/88f278fcce4348fc8d8ddf4a5fb28ef0` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:37.090099 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88f278fcce4348fc8d8ddf4a5fb28ef0)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:37.093008 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:37.147834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88f278fcce4348fc8d8ddf4a5fb28ef0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:37.163632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88f278fcce4348fc8d8ddf4a5fb28ef0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:37.178395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88f278fcce4348fc8d8ddf4a5fb28ef0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:37.179095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88f278fcce4348fc8d8ddf4a5fb28ef0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:37.187306 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88f278fcce4348fc8d8ddf4a5fb28ef0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:37.193563 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1289/2595] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:36 2022] DELETE /identity/v3/users/88f278fcce4348fc8d8ddf4a5fb28ef0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:37.198067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcc0dc-7bcf-4746-bbe5-a9564cd30286 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:37.198352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcc0dc-7bcf-4746-bbe5-a9564cd30286 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:37.198632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcc0dc-7bcf-4746-bbe5-a9564cd30286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:37.217298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-82fcc0dc-7bcf-4746-bbe5-a9564cd30286 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:37.218472 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-82fcc0dc-7bcf-4746-bbe5-a9564cd30286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:37.283389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1307/2596] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:37.293296 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:37.295634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:37.371208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e26a13-bd9c-4c6d-aac4-19d2c6c68be3 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:37.375935 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32d6be-31f0-46d1-b806-50d258decc41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:37.376657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32d6be-31f0-46d1-b806-50d258decc41 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:37.376875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32d6be-31f0-46d1-b806-50d258decc41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:37.377147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32d6be-31f0-46d1-b806-50d258decc41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:37.391015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32d6be-31f0-46d1-b806-50d258decc41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:37.393158 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32d6be-31f0-46d1-b806-50d258decc41 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:37.410861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1290/2597] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:24:37.757011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcc0dc-7bcf-4746-bbe5-a9564cd30286 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:37.758533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-82fcc0dc-7bcf-4746-bbe5-a9564cd30286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:37.778108 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcc0dc-7bcf-4746-bbe5-a9564cd30286 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:37.779445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a0c83-f999-40fd-984a-1362768e8f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:37.780665 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a0c83-f999-40fd-984a-1362768e8f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:37.780957 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a0c83-f999-40fd-984a-1362768e8f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:37.781235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a0c83-f999-40fd-984a-1362768e8f6b admin admin] PATH_INFO: `/v3/projects/c7e89a6394714440939a97a168f8105a` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:37.782487 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a0c83-f999-40fd-984a-1362768e8f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7e89a6394714440939a97a168f8105a)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:37.784417 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a0c83-f999-40fd-984a-1362768e8f6b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:37.793965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:37.838736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f74a0c83-f999-40fd-984a-1362768e8f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7e89a6394714440939a97a168f8105a', 'request_id': 'req-f74a0c83-f999-40fd-984a-1362768e8f6b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:37.845931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f74a0c83-f999-40fd-984a-1362768e8f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7e89a6394714440939a97a168f8105a', 'request_id': 'req-f74a0c83-f999-40fd-984a-1362768e8f6b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:37.851509 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f74a0c83-f999-40fd-984a-1362768e8f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7e89a6394714440939a97a168f8105a', 'request_id': 'req-f74a0c83-f999-40fd-984a-1362768e8f6b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:37.865891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f74a0c83-f999-40fd-984a-1362768e8f6b admin admin] The token cache is being invalidate because project c7e89a6394714440939a97a168f8105a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:37.866363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f74a0c83-f999-40fd-984a-1362768e8f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:37.867086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f74a0c83-f999-40fd-984a-1362768e8f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:37.869283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1308/2598] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:37 2022] DELETE /identity/v3/projects/c7e89a6394714440939a97a168f8105a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:37.875482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15f579-efa5-4dce-9a0c-505ef4e53686 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:37.876213 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15f579-efa5-4dce-9a0c-505ef4e53686 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:37.876213 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15f579-efa5-4dce-9a0c-505ef4e53686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:37.895581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-8b15f579-efa5-4dce-9a0c-505ef4e53686 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:37.896911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8b15f579-efa5-4dce-9a0c-505ef4e53686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:37.951891 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:37.967301 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1291/2599] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:37.976891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43b964-4ad2-4ea5-a984-20b5c3c92106 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:37.977183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43b964-4ad2-4ea5-a984-20b5c3c92106 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:37.977459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43b964-4ad2-4ea5-a984-20b5c3c92106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:37.996524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5e43b964-4ad2-4ea5-a984-20b5c3c92106 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:37.997906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5e43b964-4ad2-4ea5-a984-20b5c3c92106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:38.005833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1309/2600] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:38.010555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15f579-efa5-4dce-9a0c-505ef4e53686 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:38.020649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15f579-efa5-4dce-9a0c-505ef4e53686 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:38.025505 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc721-bdb5-40f7-a2c9-f14844b5d133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:38.026269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc721-bdb5-40f7-a2c9-f14844b5d133 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.026616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc721-bdb5-40f7-a2c9-f14844b5d133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.026958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc721-bdb5-40f7-a2c9-f14844b5d133 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.028016 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bc721-bdb5-40f7-a2c9-f14844b5d133 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:38.029960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bc721-bdb5-40f7-a2c9-f14844b5d133 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:38.031641 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bc721-bdb5-40f7-a2c9-f14844b5d133 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:38.048372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1292/2601] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:24:38.049638 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43b964-4ad2-4ea5-a984-20b5c3c92106 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:38.060656 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43b964-4ad2-4ea5-a984-20b5c3c92106 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:38.065544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84ea73-2b08-4dfc-94df-934fc2a8c2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:38.066271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84ea73-2b08-4dfc-94df-934fc2a8c2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.066552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84ea73-2b08-4dfc-94df-934fc2a8c2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.066924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84ea73-2b08-4dfc-94df-934fc2a8c2b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.067610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84ea73-2b08-4dfc-94df-934fc2a8c2b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:38.069509 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84ea73-2b08-4dfc-94df-934fc2a8c2b7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:38.098107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1310/2602] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:38 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:38.103090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc721-bdb5-40f7-a2c9-f14844b5d133 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:38.115692 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc721-bdb5-40f7-a2c9-f14844b5d133 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:38.187539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1c80a-9224-4c1e-beed-05e8f95e7043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:38.188207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1c80a-9224-4c1e-beed-05e8f95e7043 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.188486 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1c80a-9224-4c1e-beed-05e8f95e7043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.188756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1c80a-9224-4c1e-beed-05e8f95e7043 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.189608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1c80a-9224-4c1e-beed-05e8f95e7043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:38.191832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1c80a-9224-4c1e-beed-05e8f95e7043 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:38.227121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1293/2603] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:38 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:38.232172 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84ea73-2b08-4dfc-94df-934fc2a8c2b7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:38.244410 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84ea73-2b08-4dfc-94df-934fc2a8c2b7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:38.249975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0d732-4327-4884-91a1-4dbe5d730ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:38.250664 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0d732-4327-4884-91a1-4dbe5d730ada admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.250945 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0d732-4327-4884-91a1-4dbe5d730ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.251217 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0d732-4327-4884-91a1-4dbe5d730ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.252500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0d732-4327-4884-91a1-4dbe5d730ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:38.255087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0d732-4327-4884-91a1-4dbe5d730ada admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:38.271672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1311/2604] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:38.276332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1c80a-9224-4c1e-beed-05e8f95e7043 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:38.289332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1c80a-9224-4c1e-beed-05e8f95e7043 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:38.294595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c2d89-30bc-4244-a1dc-20e0e58ff073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:38.295305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c2d89-30bc-4244-a1dc-20e0e58ff073 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.295590 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c2d89-30bc-4244-a1dc-20e0e58ff073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.295884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c2d89-30bc-4244-a1dc-20e0e58ff073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.296611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c2d89-30bc-4244-a1dc-20e0e58ff073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:38.298241 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c2d89-30bc-4244-a1dc-20e0e58ff073 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:38.312648 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1294/2605] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:38.317255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0d732-4327-4884-91a1-4dbe5d730ada admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:38.328548 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0d732-4327-4884-91a1-4dbe5d730ada admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:38.333385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b17a97-fb6e-4640-b27c-ca4fd0621eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:38.334355 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b17a97-fb6e-4640-b27c-ca4fd0621eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.334635 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b17a97-fb6e-4640-b27c-ca4fd0621eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.334906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b17a97-fb6e-4640-b27c-ca4fd0621eb7 admin admin] PATH_INFO: `/v3/projects/b13f7a47b81044d2895b60f26a8869b1/users/4c38335cd9c44287bf5a8f246cf6366f/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.351280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b17a97-fb6e-4640-b27c-ca4fd0621eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b13f7a47b81044d2895b60f26a8869b1, user_id=4c38335cd9c44287bf5a8f246cf6366f, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:38.354019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b17a97-fb6e-4640-b27c-ca4fd0621eb7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:38.368068 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1312/2606] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:38 2022] PUT /identity/v3/projects/b13f7a47b81044d2895b60f26a8869b1/users/4c38335cd9c44287bf5a8f246cf6366f/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:38.379695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b1fef-463e-4786-aa92-e407ddcb8828 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.379981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b1fef-463e-4786-aa92-e407ddcb8828 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.380258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b1fef-463e-4786-aa92-e407ddcb8828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.408665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b02b1fef-463e-4786-aa92-e407ddcb8828 None None] MFA Rules not processed for user `4c38335cd9c44287bf5a8f246cf6366f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:38.409904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b02b1fef-463e-4786-aa92-e407ddcb8828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:38.479945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1295/2607] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:38.485606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b17a97-fb6e-4640-b27c-ca4fd0621eb7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:38.553159 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b17a97-fb6e-4640-b27c-ca4fd0621eb7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:38.558500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380dbc9d-9daf-4856-a704-0ac94ea1e43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:38.559156 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380dbc9d-9daf-4856-a704-0ac94ea1e43d admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.559465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380dbc9d-9daf-4856-a704-0ac94ea1e43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.559776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380dbc9d-9daf-4856-a704-0ac94ea1e43d admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.560648 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380dbc9d-9daf-4856-a704-0ac94ea1e43d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:38.562626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380dbc9d-9daf-4856-a704-0ac94ea1e43d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:38.592729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1313/2608] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:38 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:38.597532 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b1fef-463e-4786-aa92-e407ddcb8828 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:38.611202 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b1fef-463e-4786-aa92-e407ddcb8828 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:38.664435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:38.680005 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe117c-aa60-40cf-9376-cff462a360f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:38.680984 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe117c-aa60-40cf-9376-cff462a360f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.681483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe117c-aa60-40cf-9376-cff462a360f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.682032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe117c-aa60-40cf-9376-cff462a360f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.682763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe117c-aa60-40cf-9376-cff462a360f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:38.684747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe117c-aa60-40cf-9376-cff462a360f3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:38.705911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1296/2609] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:38 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:38.710330 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380dbc9d-9daf-4856-a704-0ac94ea1e43d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:38.723165 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380dbc9d-9daf-4856-a704-0ac94ea1e43d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:38.729502 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc53e1-f733-4b90-99f1-2e8e8c8ff712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:38.730224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc53e1-f733-4b90-99f1-2e8e8c8ff712 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.730502 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc53e1-f733-4b90-99f1-2e8e8c8ff712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.730777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc53e1-f733-4b90-99f1-2e8e8c8ff712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.731504 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc53e1-f733-4b90-99f1-2e8e8c8ff712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:38.734017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc53e1-f733-4b90-99f1-2e8e8c8ff712 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:38.748699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1314/2610] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:38.753971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe117c-aa60-40cf-9376-cff462a360f3 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:38.764286 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe117c-aa60-40cf-9376-cff462a360f3 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:38.769257 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb954394-883a-4b6c-8328-82c3b6c1ea01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:38.770186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb954394-883a-4b6c-8328-82c3b6c1ea01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.770464 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb954394-883a-4b6c-8328-82c3b6c1ea01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.770730 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb954394-883a-4b6c-8328-82c3b6c1ea01 admin admin] PATH_INFO: `/v3/projects/67d29663a58f4439840dab3df390a319/users/42d6935fa3f7434d8da0b6ab57064270/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.787898 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb954394-883a-4b6c-8328-82c3b6c1ea01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d29663a58f4439840dab3df390a319, user_id=42d6935fa3f7434d8da0b6ab57064270, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:38.791192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb954394-883a-4b6c-8328-82c3b6c1ea01 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:38.806198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1297/2611] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:38 2022] PUT /identity/v3/projects/67d29663a58f4439840dab3df390a319/users/42d6935fa3f7434d8da0b6ab57064270/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:38.811750 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc53e1-f733-4b90-99f1-2e8e8c8ff712 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:38.823825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc53e1-f733-4b90-99f1-2e8e8c8ff712 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:38.830130 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:38.895461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c195f6c-443c-4b47-ac20-f93c1ba63354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:38.896141 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c195f6c-443c-4b47-ac20-f93c1ba63354 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.896423 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c195f6c-443c-4b47-ac20-f93c1ba63354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.896695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c195f6c-443c-4b47-ac20-f93c1ba63354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.897419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c195f6c-443c-4b47-ac20-f93c1ba63354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:38.899404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c195f6c-443c-4b47-ac20-f93c1ba63354 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:38.912275 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1315/2612] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:38.916701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb954394-883a-4b6c-8328-82c3b6c1ea01 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:38.928509 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb954394-883a-4b6c-8328-82c3b6c1ea01 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:38.933159 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df0703-a1a1-4ef5-94a8-8244bc1a0d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:38.934074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df0703-a1a1-4ef5-94a8-8244bc1a0d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.934343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df0703-a1a1-4ef5-94a8-8244bc1a0d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.934606 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df0703-a1a1-4ef5-94a8-8244bc1a0d44 admin admin] PATH_INFO: `/v3/projects/67d29663a58f4439840dab3df390a319/users/42d6935fa3f7434d8da0b6ab57064270/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.937011 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df0703-a1a1-4ef5-94a8-8244bc1a0d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d29663a58f4439840dab3df390a319, user_id=42d6935fa3f7434d8da0b6ab57064270, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:38.940618 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df0703-a1a1-4ef5-94a8-8244bc1a0d44 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:38.952321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1298/2613] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:38 2022] PUT /identity/v3/projects/67d29663a58f4439840dab3df390a319/users/42d6935fa3f7434d8da0b6ab57064270/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:38.962835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7acad6-1725-4867-9642-5ab0e1f14be9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:38.963388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7acad6-1725-4867-9642-5ab0e1f14be9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:38.963662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7acad6-1725-4867-9642-5ab0e1f14be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:38.992788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-bf7acad6-1725-4867-9642-5ab0e1f14be9 None None] MFA Rules not processed for user `42d6935fa3f7434d8da0b6ab57064270`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:38.994046 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bf7acad6-1725-4867-9642-5ab0e1f14be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:39.082060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1316/2614] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:39.095741 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df0703-a1a1-4ef5-94a8-8244bc1a0d44 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:39.097486 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-80df0703-a1a1-4ef5-94a8-8244bc1a0d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:39.163035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df0703-a1a1-4ef5-94a8-8244bc1a0d44 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:39.166867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5b881-7526-46c1-a212-4b47377c5b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:39.167951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5b881-7526-46c1-a212-4b47377c5b6e service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:39.168355 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5b881-7526-46c1-a212-4b47377c5b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:39.168744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5b881-7526-46c1-a212-4b47377c5b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:39.179285 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5b881-7526-46c1-a212-4b47377c5b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:39.180727 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5b881-7526-46c1-a212-4b47377c5b6e service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:39.191469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1299/2615] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:39.220589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7acad6-1725-4867-9642-5ab0e1f14be9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:39.291021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7acad6-1725-4867-9642-5ab0e1f14be9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:39.297978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:39.299435 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:39.299850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:39.300135 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] PATH_INFO: `/v3/users/4c38335cd9c44287bf5a8f246cf6366f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:39.301716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c38335cd9c44287bf5a8f246cf6366f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:39.305083 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:39.357971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:39.379374 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c38335cd9c44287bf5a8f246cf6366f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:39.394613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c38335cd9c44287bf5a8f246cf6366f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:39.394613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c38335cd9c44287bf5a8f246cf6366f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:39.394613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c38335cd9c44287bf5a8f246cf6366f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:39.409298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c38335cd9c44287bf5a8f246cf6366f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:39.422511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1317/2616] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:39 2022] DELETE /identity/v3/users/4c38335cd9c44287bf5a8f246cf6366f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:39.427258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3681cc-4b59-47f1-aa7b-bc21f4135a2e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:39.427571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3681cc-4b59-47f1-aa7b-bc21f4135a2e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:39.427849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3681cc-4b59-47f1-aa7b-bc21f4135a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:39.449092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2a3681cc-4b59-47f1-aa7b-bc21f4135a2e None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:39.450965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2a3681cc-4b59-47f1-aa7b-bc21f4135a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:39.519171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1300/2617] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:39.528806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:39.530430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:39.596257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4011ee-db2c-4449-ae9f-161ff036dd8f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:39.600915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056fb15-9b97-4cfa-9a6d-eec26b87ed16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:39.601563 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056fb15-9b97-4cfa-9a6d-eec26b87ed16 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:39.602017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056fb15-9b97-4cfa-9a6d-eec26b87ed16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:39.602123 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056fb15-9b97-4cfa-9a6d-eec26b87ed16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:39.614121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056fb15-9b97-4cfa-9a6d-eec26b87ed16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:39.615576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056fb15-9b97-4cfa-9a6d-eec26b87ed16 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:39.626689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1318/2618] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:39.931207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3681cc-4b59-47f1-aa7b-bc21f4135a2e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:39.933178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2a3681cc-4b59-47f1-aa7b-bc21f4135a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:39.945443 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3681cc-4b59-47f1-aa7b-bc21f4135a2e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:39.950053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:39.951184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:39.951460 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:39.952175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] PATH_INFO: `/v3/projects/b13f7a47b81044d2895b60f26a8869b1` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:39.953024 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b13f7a47b81044d2895b60f26a8869b1)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:39.955104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:39.996881 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:40.007766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b13f7a47b81044d2895b60f26a8869b1', 'request_id': 'req-1ee0c6f3-4b0f-458b-83e4-411435787cf5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.016958 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b13f7a47b81044d2895b60f26a8869b1', 'request_id': 'req-1ee0c6f3-4b0f-458b-83e4-411435787cf5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.023264 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b13f7a47b81044d2895b60f26a8869b1', 'request_id': 'req-1ee0c6f3-4b0f-458b-83e4-411435787cf5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.035635 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] The token cache is being invalidate because project b13f7a47b81044d2895b60f26a8869b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:40.035925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.036602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.038207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1301/2619] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:39 2022] DELETE /identity/v3/projects/b13f7a47b81044d2895b60f26a8869b1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:40.042329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056fb15-9b97-4cfa-9a6d-eec26b87ed16 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:40.053825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056fb15-9b97-4cfa-9a6d-eec26b87ed16 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:40.112371 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff96856-c046-41fc-b1f3-e402b74ed2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:40.113309 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff96856-c046-41fc-b1f3-e402b74ed2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:40.113600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff96856-c046-41fc-b1f3-e402b74ed2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:40.113924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff96856-c046-41fc-b1f3-e402b74ed2fb admin admin] PATH_INFO: `/v3/users/42d6935fa3f7434d8da0b6ab57064270` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:40.115212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff96856-c046-41fc-b1f3-e402b74ed2fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=42d6935fa3f7434d8da0b6ab57064270)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:40.117162 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff96856-c046-41fc-b1f3-e402b74ed2fb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:40.165587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2ff96856-c046-41fc-b1f3-e402b74ed2fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42d6935fa3f7434d8da0b6ab57064270'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.169824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:40.174334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2ff96856-c046-41fc-b1f3-e402b74ed2fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42d6935fa3f7434d8da0b6ab57064270'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.175476 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2ff96856-c046-41fc-b1f3-e402b74ed2fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42d6935fa3f7434d8da0b6ab57064270'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.176305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2ff96856-c046-41fc-b1f3-e402b74ed2fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42d6935fa3f7434d8da0b6ab57064270'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.190616 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2ff96856-c046-41fc-b1f3-e402b74ed2fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42d6935fa3f7434d8da0b6ab57064270'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.203044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1319/2620] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:40 2022] DELETE /identity/v3/users/42d6935fa3f7434d8da0b6ab57064270 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:40.478191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:40.479696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:40.493185 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0c6f3-4b0f-458b-83e4-411435787cf5 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:40.556505 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493df8b3-25fc-46d8-a427-e4b3ba6cf951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:40.557325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493df8b3-25fc-46d8-a427-e4b3ba6cf951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:40.557602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493df8b3-25fc-46d8-a427-e4b3ba6cf951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:40.557910 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493df8b3-25fc-46d8-a427-e4b3ba6cf951 admin admin] PATH_INFO: `/v3/projects/67d29663a58f4439840dab3df390a319` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:40.559094 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493df8b3-25fc-46d8-a427-e4b3ba6cf951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67d29663a58f4439840dab3df390a319)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:40.561207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493df8b3-25fc-46d8-a427-e4b3ba6cf951 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:40.609750 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-493df8b3-25fc-46d8-a427-e4b3ba6cf951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67d29663a58f4439840dab3df390a319', 'request_id': 'req-493df8b3-25fc-46d8-a427-e4b3ba6cf951'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.617703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-493df8b3-25fc-46d8-a427-e4b3ba6cf951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67d29663a58f4439840dab3df390a319', 'request_id': 'req-493df8b3-25fc-46d8-a427-e4b3ba6cf951'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.624164 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-493df8b3-25fc-46d8-a427-e4b3ba6cf951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67d29663a58f4439840dab3df390a319', 'request_id': 'req-493df8b3-25fc-46d8-a427-e4b3ba6cf951'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.634383 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-493df8b3-25fc-46d8-a427-e4b3ba6cf951 admin admin] The token cache is being invalidate because project 67d29663a58f4439840dab3df390a319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:40.634743 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-493df8b3-25fc-46d8-a427-e4b3ba6cf951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.635309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-493df8b3-25fc-46d8-a427-e4b3ba6cf951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:40.636864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1302/2621] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:40 2022] DELETE /identity/v3/projects/67d29663a58f4439840dab3df390a319 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:40.642646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca7b7a-3a64-4689-b7c4-46351080d850 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:40.642928 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca7b7a-3a64-4689-b7c4-46351080d850 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:40.643203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca7b7a-3a64-4689-b7c4-46351080d850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:40.662399 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-51ca7b7a-3a64-4689-b7c4-46351080d850 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:40.663605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-51ca7b7a-3a64-4689-b7c4-46351080d850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:40.723404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1320/2622] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:40.732802 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3965ad4-741d-4fcf-87e3-2a77817de98c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:40.733088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3965ad4-741d-4fcf-87e3-2a77817de98c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:40.733362 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3965ad4-741d-4fcf-87e3-2a77817de98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:40.752003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b3965ad4-741d-4fcf-87e3-2a77817de98c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:40.753191 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b3965ad4-741d-4fcf-87e3-2a77817de98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:40.761049 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1303/2623] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:40.766295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca7b7a-3a64-4689-b7c4-46351080d850 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:40.776220 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca7b7a-3a64-4689-b7c4-46351080d850 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:40.780893 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bbb6d-84d3-41a5-893b-c4a10acf1f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:40.781535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bbb6d-84d3-41a5-893b-c4a10acf1f85 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:40.781853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bbb6d-84d3-41a5-893b-c4a10acf1f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:40.782377 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bbb6d-84d3-41a5-893b-c4a10acf1f85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:40.783138 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bbb6d-84d3-41a5-893b-c4a10acf1f85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:40.783443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bbb6d-84d3-41a5-893b-c4a10acf1f85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:40.785060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bbb6d-84d3-41a5-893b-c4a10acf1f85 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:40.797524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1321/2624] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:24:40.801835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3965ad4-741d-4fcf-87e3-2a77817de98c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:40.812196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3965ad4-741d-4fcf-87e3-2a77817de98c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:40.816861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2de7f-76e5-49ff-aeb5-89b2a6181d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:40.817524 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2de7f-76e5-49ff-aeb5-89b2a6181d38 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:40.817835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2de7f-76e5-49ff-aeb5-89b2a6181d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:40.818111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2de7f-76e5-49ff-aeb5-89b2a6181d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:40.818790 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2de7f-76e5-49ff-aeb5-89b2a6181d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:40.820686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2de7f-76e5-49ff-aeb5-89b2a6181d38 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:40.848712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1304/2625] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:40 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:40.852342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bbb6d-84d3-41a5-893b-c4a10acf1f85 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:40.862706 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bbb6d-84d3-41a5-893b-c4a10acf1f85 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:40.920694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dcd66-a27c-4351-bc62-9176850a2a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:40.921369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dcd66-a27c-4351-bc62-9176850a2a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:40.921653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dcd66-a27c-4351-bc62-9176850a2a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:40.921965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dcd66-a27c-4351-bc62-9176850a2a04 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:40.922658 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dcd66-a27c-4351-bc62-9176850a2a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:40.925395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dcd66-a27c-4351-bc62-9176850a2a04 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:40.953914 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1322/2626] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:40 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:40.958003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2de7f-76e5-49ff-aeb5-89b2a6181d38 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:40.968509 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2de7f-76e5-49ff-aeb5-89b2a6181d38 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:40.973103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a42edf-e3fa-42e1-8fed-573817aab467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:40.973782 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a42edf-e3fa-42e1-8fed-573817aab467 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:40.974100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a42edf-e3fa-42e1-8fed-573817aab467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:40.974449 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a42edf-e3fa-42e1-8fed-573817aab467 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:40.975223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a42edf-e3fa-42e1-8fed-573817aab467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:40.976912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a42edf-e3fa-42e1-8fed-573817aab467 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:40.988412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1305/2627] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:40.992427 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dcd66-a27c-4351-bc62-9176850a2a04 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:41.003880 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dcd66-a27c-4351-bc62-9176850a2a04 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:41.008572 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c553a1-f3c6-4fc1-aeb6-32208fb9274f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:41.009293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c553a1-f3c6-4fc1-aeb6-32208fb9274f admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:41.009579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c553a1-f3c6-4fc1-aeb6-32208fb9274f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:41.009897 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c553a1-f3c6-4fc1-aeb6-32208fb9274f admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:41.010624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c553a1-f3c6-4fc1-aeb6-32208fb9274f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:41.012267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c553a1-f3c6-4fc1-aeb6-32208fb9274f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:41.023517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1323/2628] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:41.027859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a42edf-e3fa-42e1-8fed-573817aab467 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:41.038010 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a42edf-e3fa-42e1-8fed-573817aab467 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:41.042634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e98a4e-66f8-4ed4-8c5e-68e509a766da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:41.043499 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e98a4e-66f8-4ed4-8c5e-68e509a766da admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:41.043777 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e98a4e-66f8-4ed4-8c5e-68e509a766da admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:41.044572 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e98a4e-66f8-4ed4-8c5e-68e509a766da admin admin] PATH_INFO: `/v3/projects/cff33b33a65a4adcb3e562f78f34a30e/users/f06d05d7251d4e908914c69ebb57019e/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:41.060786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e98a4e-66f8-4ed4-8c5e-68e509a766da admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff33b33a65a4adcb3e562f78f34a30e, user_id=f06d05d7251d4e908914c69ebb57019e, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:41.063553 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e98a4e-66f8-4ed4-8c5e-68e509a766da admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:41.065843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:41.078385 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1306/2629] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:41 2022] PUT /identity/v3/projects/cff33b33a65a4adcb3e562f78f34a30e/users/f06d05d7251d4e908914c69ebb57019e/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:41.088415 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa982f7-120c-4b40-aabf-af5d106b1811 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:41.088917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa982f7-120c-4b40-aabf-af5d106b1811 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:41.089014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa982f7-120c-4b40-aabf-af5d106b1811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:41.118653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-aaa982f7-120c-4b40-aabf-af5d106b1811 None None] MFA Rules not processed for user `f06d05d7251d4e908914c69ebb57019e`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:41.119894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-aaa982f7-120c-4b40-aabf-af5d106b1811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:41.145070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:41.194004 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1324/2630] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:41.198478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e98a4e-66f8-4ed4-8c5e-68e509a766da admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:41.263222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e98a4e-66f8-4ed4-8c5e-68e509a766da admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:41.267926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14192e9-f829-4db9-a71b-2e0742a01973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:41.268583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14192e9-f829-4db9-a71b-2e0742a01973 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:41.268864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14192e9-f829-4db9-a71b-2e0742a01973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:41.269137 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14192e9-f829-4db9-a71b-2e0742a01973 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:41.269877 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14192e9-f829-4db9-a71b-2e0742a01973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:41.271791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14192e9-f829-4db9-a71b-2e0742a01973 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:41.298415 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1307/2631] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:41 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:41.302720 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa982f7-120c-4b40-aabf-af5d106b1811 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:41.314178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa982f7-120c-4b40-aabf-af5d106b1811 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:41.374001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c345b-0b66-47c8-bd65-59ff66ee487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:41.374667 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c345b-0b66-47c8-bd65-59ff66ee487a admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:41.374949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c345b-0b66-47c8-bd65-59ff66ee487a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:41.375224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c345b-0b66-47c8-bd65-59ff66ee487a admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:41.375903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c345b-0b66-47c8-bd65-59ff66ee487a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:41.377898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c345b-0b66-47c8-bd65-59ff66ee487a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:41.398624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1325/2632] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:41 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:24:41.402621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14192e9-f829-4db9-a71b-2e0742a01973 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:41.413473 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14192e9-f829-4db9-a71b-2e0742a01973 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:41.418478 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a8982-ac9a-4ab2-af9f-4737a607d754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:41.419912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a8982-ac9a-4ab2-af9f-4737a607d754 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:41.420150 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a8982-ac9a-4ab2-af9f-4737a607d754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:41.420430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a8982-ac9a-4ab2-af9f-4737a607d754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:41.421145 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a8982-ac9a-4ab2-af9f-4737a607d754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:41.422775 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a8982-ac9a-4ab2-af9f-4737a607d754 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:41.434115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1308/2633] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:41.438532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c345b-0b66-47c8-bd65-59ff66ee487a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:41.448565 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c345b-0b66-47c8-bd65-59ff66ee487a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:41.453595 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8f0f7-80a7-4b42-bdb3-8ff3f549ae4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:41.454479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8f0f7-80a7-4b42-bdb3-8ff3f549ae4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:41.454759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8f0f7-80a7-4b42-bdb3-8ff3f549ae4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:41.455026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8f0f7-80a7-4b42-bdb3-8ff3f549ae4a admin admin] PATH_INFO: `/v3/projects/d272b3bb14e745cf978b222c4a756a0e/users/64ecede0439c4e8db7e4d5ed990b3d28/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:41.471064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8f0f7-80a7-4b42-bdb3-8ff3f549ae4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d272b3bb14e745cf978b222c4a756a0e, user_id=64ecede0439c4e8db7e4d5ed990b3d28, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:41.473800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8f0f7-80a7-4b42-bdb3-8ff3f549ae4a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:41.487514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1326/2634] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:41 2022] PUT /identity/v3/projects/d272b3bb14e745cf978b222c4a756a0e/users/64ecede0439c4e8db7e4d5ed990b3d28/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:41.491623 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a8982-ac9a-4ab2-af9f-4737a607d754 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:41.502102 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a8982-ac9a-4ab2-af9f-4737a607d754 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:41.564545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756aafd7-1479-44ed-91b3-97cf1ec0158d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:41.565131 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756aafd7-1479-44ed-91b3-97cf1ec0158d admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:41.565416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756aafd7-1479-44ed-91b3-97cf1ec0158d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:41.565690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756aafd7-1479-44ed-91b3-97cf1ec0158d admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:41.566519 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756aafd7-1479-44ed-91b3-97cf1ec0158d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:41.568126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756aafd7-1479-44ed-91b3-97cf1ec0158d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:41.578564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1309/2635] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:24:41.582821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8f0f7-80a7-4b42-bdb3-8ff3f549ae4a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:41.594078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8f0f7-80a7-4b42-bdb3-8ff3f549ae4a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:41.598795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d83ef-fd8e-4bdd-9c38-88a6d90aa9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:41.599682 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d83ef-fd8e-4bdd-9c38-88a6d90aa9d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:41.599963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d83ef-fd8e-4bdd-9c38-88a6d90aa9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:41.600234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d83ef-fd8e-4bdd-9c38-88a6d90aa9d1 admin admin] PATH_INFO: `/v3/projects/d272b3bb14e745cf978b222c4a756a0e/users/64ecede0439c4e8db7e4d5ed990b3d28/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:41.602274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d83ef-fd8e-4bdd-9c38-88a6d90aa9d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d272b3bb14e745cf978b222c4a756a0e, user_id=64ecede0439c4e8db7e4d5ed990b3d28, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:41.604930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d83ef-fd8e-4bdd-9c38-88a6d90aa9d1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:41.616202 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1327/2636] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:41 2022] PUT /identity/v3/projects/d272b3bb14e745cf978b222c4a756a0e/users/64ecede0439c4e8db7e4d5ed990b3d28/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:41.627729 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd970d26-0102-4cb9-a9d3-5eef5a64d539 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:41.628014 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd970d26-0102-4cb9-a9d3-5eef5a64d539 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:41.628307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd970d26-0102-4cb9-a9d3-5eef5a64d539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:41.656189 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bd970d26-0102-4cb9-a9d3-5eef5a64d539 None None] MFA Rules not processed for user `64ecede0439c4e8db7e4d5ed990b3d28`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:41.657390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bd970d26-0102-4cb9-a9d3-5eef5a64d539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:41.708338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:41.729603 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1310/2637] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:41.742265 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d83ef-fd8e-4bdd-9c38-88a6d90aa9d1 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:41.743985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0e8d83ef-fd8e-4bdd-9c38-88a6d90aa9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:41.803601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d83ef-fd8e-4bdd-9c38-88a6d90aa9d1 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:41.807213 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ba7d8-d1af-4612-bebc-0173e1757752 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:41.807853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ba7d8-d1af-4612-bebc-0173e1757752 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:41.808133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ba7d8-d1af-4612-bebc-0173e1757752 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:41.808404 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ba7d8-d1af-4612-bebc-0173e1757752 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:41.818825 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ba7d8-d1af-4612-bebc-0173e1757752 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:41.820238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ba7d8-d1af-4612-bebc-0173e1757752 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:41.829882 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1328/2638] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:42.148891 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd970d26-0102-4cb9-a9d3-5eef5a64d539 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:42.159363 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd970d26-0102-4cb9-a9d3-5eef5a64d539 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:42.164320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809935bf-96de-4b27-949d-720f5af7aa96 tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ecede0439c4e8db7e4d5ed990b3d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd272b3bb14e745cf978b222c4a756a0e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:42.164995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809935bf-96de-4b27-949d-720f5af7aa96 tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:42.165429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809935bf-96de-4b27-949d-720f5af7aa96 tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:42.165714 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809935bf-96de-4b27-949d-720f5af7aa96 tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:42.166481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809935bf-96de-4b27-949d-720f5af7aa96 tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:42.166799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809935bf-96de-4b27-949d-720f5af7aa96 tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:42.168390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809935bf-96de-4b27-949d-720f5af7aa96 tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:42.181566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1311/2639] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:24:42.185601 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ba7d8-d1af-4612-bebc-0173e1757752 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:42.194614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ba7d8-d1af-4612-bebc-0173e1757752 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:42.204650 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007c894-5913-44b5-9855-2628bd302f2a tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ecede0439c4e8db7e4d5ed990b3d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd272b3bb14e745cf978b222c4a756a0e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:42.205294 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007c894-5913-44b5-9855-2628bd302f2a tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:42.205573 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007c894-5913-44b5-9855-2628bd302f2a tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:42.205871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007c894-5913-44b5-9855-2628bd302f2a tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:42.206552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007c894-5913-44b5-9855-2628bd302f2a tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:42.208468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007c894-5913-44b5-9855-2628bd302f2a tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:42.228525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:42.236901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1329/2640] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:42 2022] POST /identity/v3/projects => generated 355 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:42.241118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809935bf-96de-4b27-949d-720f5af7aa96 tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:42.251477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809935bf-96de-4b27-949d-720f5af7aa96 tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:42.313517 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11723d20-322e-42fa-abbe-85d3cca76c2c tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ecede0439c4e8db7e4d5ed990b3d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd272b3bb14e745cf978b222c4a756a0e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:42.314252 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11723d20-322e-42fa-abbe-85d3cca76c2c tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:42.314566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11723d20-322e-42fa-abbe-85d3cca76c2c tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:42.314846 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11723d20-322e-42fa-abbe-85d3cca76c2c tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:42.315577 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11723d20-322e-42fa-abbe-85d3cca76c2c tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:42.315890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11723d20-322e-42fa-abbe-85d3cca76c2c tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:42.317491 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11723d20-322e-42fa-abbe-85d3cca76c2c tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:42.329839 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1312/2641] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:24:42.367292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007c894-5913-44b5-9855-2628bd302f2a tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:42.378391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007c894-5913-44b5-9855-2628bd302f2a tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:42.383093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64ecede0439c4e8db7e4d5ed990b3d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd272b3bb14e745cf978b222c4a756a0e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:42.383942 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:42.384228 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:42.384506 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] PATH_INFO: `/v3/projects/cbb32c6d9ce94e7bb9965dcb15075732` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:42.385676 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cbb32c6d9ce94e7bb9965dcb15075732)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:42.387622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:42.437305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb32c6d9ce94e7bb9965dcb15075732', 'request_id': 'req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:42.444462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb32c6d9ce94e7bb9965dcb15075732', 'request_id': 'req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:42.450268 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb32c6d9ce94e7bb9965dcb15075732', 'request_id': 'req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:42.462966 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] The token cache is being invalidate because project cbb32c6d9ce94e7bb9965dcb15075732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:42.463255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:42.463896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-58f62db0-84a4-4bc1-8c1f-e516564cfa7f tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:42.465477 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1330/2642] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:42 2022] DELETE /identity/v3/projects/cbb32c6d9ce94e7bb9965dcb15075732 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:42.668326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11723d20-322e-42fa-abbe-85d3cca76c2c tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:42.670406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-11723d20-322e-42fa-abbe-85d3cca76c2c tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:42.737786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11723d20-322e-42fa-abbe-85d3cca76c2c tempest-VolumeQuotaClassesTest-267692353 tempest-VolumeQuotaClassesTest-267692353-project-admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:42.806130 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:42.807080 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:42.807408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:42.807731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] PATH_INFO: `/v3/users/f06d05d7251d4e908914c69ebb57019e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:42.809049 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f06d05d7251d4e908914c69ebb57019e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:42.811561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:42.835349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:42.865350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f06d05d7251d4e908914c69ebb57019e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:42.878057 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f06d05d7251d4e908914c69ebb57019e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:42.893320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f06d05d7251d4e908914c69ebb57019e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:42.894097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f06d05d7251d4e908914c69ebb57019e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:42.901608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f06d05d7251d4e908914c69ebb57019e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:42.908728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1313/2643] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:42 2022] DELETE /identity/v3/users/f06d05d7251d4e908914c69ebb57019e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:42.913380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3b52d-1885-4d16-a785-5ac8d350785f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:42.913664 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3b52d-1885-4d16-a785-5ac8d350785f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:42.913972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3b52d-1885-4d16-a785-5ac8d350785f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:42.936505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b0e3b52d-1885-4d16-a785-5ac8d350785f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:42.937760 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b0e3b52d-1885-4d16-a785-5ac8d350785f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:43.004594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1331/2644] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:43.015126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:43.016786 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:43.081305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941376c1-e68a-47c7-8e29-05380585e335 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:43.084991 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ffec4-e2d4-40c5-b980-3e030d702a78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:43.085633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ffec4-e2d4-40c5-b980-3e030d702a78 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:43.085939 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ffec4-e2d4-40c5-b980-3e030d702a78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:43.086212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ffec4-e2d4-40c5-b980-3e030d702a78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:43.095820 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ffec4-e2d4-40c5-b980-3e030d702a78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:43.097241 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ffec4-e2d4-40c5-b980-3e030d702a78 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:43.107076 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1314/2645] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:43.401716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3b52d-1885-4d16-a785-5ac8d350785f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:43.403975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b0e3b52d-1885-4d16-a785-5ac8d350785f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:43.425576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3b52d-1885-4d16-a785-5ac8d350785f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:43.431549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:43.432422 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:43.432707 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:43.433007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] PATH_INFO: `/v3/projects/cff33b33a65a4adcb3e562f78f34a30e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:43.434235 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff33b33a65a4adcb3e562f78f34a30e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:43.436606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:43.488963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff33b33a65a4adcb3e562f78f34a30e', 'request_id': 'req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:43.495295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff33b33a65a4adcb3e562f78f34a30e', 'request_id': 'req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:43.501806 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff33b33a65a4adcb3e562f78f34a30e', 'request_id': 'req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:43.514955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] The token cache is being invalidate because project cff33b33a65a4adcb3e562f78f34a30e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:43.515242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:43.515859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:43.517449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1332/2646] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:43 2022] DELETE /identity/v3/projects/cff33b33a65a4adcb3e562f78f34a30e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:43.521941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ffec4-e2d4-40c5-b980-3e030d702a78 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:43.533627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ffec4-e2d4-40c5-b980-3e030d702a78 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:43.592237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed2f88-acb8-43a9-b3c2-bcfe13ddb74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:43.593430 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed2f88-acb8-43a9-b3c2-bcfe13ddb74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:43.593706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed2f88-acb8-43a9-b3c2-bcfe13ddb74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:43.594001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed2f88-acb8-43a9-b3c2-bcfe13ddb74b admin admin] PATH_INFO: `/v3/users/64ecede0439c4e8db7e4d5ed990b3d28` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:43.595207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed2f88-acb8-43a9-b3c2-bcfe13ddb74b admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ecede0439c4e8db7e4d5ed990b3d28)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:43.597121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed2f88-acb8-43a9-b3c2-bcfe13ddb74b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:43.641274 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:43.647711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d6ed2f88-acb8-43a9-b3c2-bcfe13ddb74b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ecede0439c4e8db7e4d5ed990b3d28'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:43.655660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d6ed2f88-acb8-43a9-b3c2-bcfe13ddb74b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ecede0439c4e8db7e4d5ed990b3d28'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:43.667377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d6ed2f88-acb8-43a9-b3c2-bcfe13ddb74b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ecede0439c4e8db7e4d5ed990b3d28'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:43.668188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d6ed2f88-acb8-43a9-b3c2-bcfe13ddb74b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ecede0439c4e8db7e4d5ed990b3d28'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:43.677487 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d6ed2f88-acb8-43a9-b3c2-bcfe13ddb74b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ecede0439c4e8db7e4d5ed990b3d28'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:43.685526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1315/2647] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:43 2022] DELETE /identity/v3/users/64ecede0439c4e8db7e4d5ed990b3d28 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:43.984293 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:43.986221 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:43.998439 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:44.004362 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132aaf1d-53ac-4623-a4a0-9fc129b7b1f9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:44.077218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4299bf-de8d-4761-88c8-93bacd0e5f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:44.078259 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4299bf-de8d-4761-88c8-93bacd0e5f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.078385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4299bf-de8d-4761-88c8-93bacd0e5f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.078969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4299bf-de8d-4761-88c8-93bacd0e5f10 admin admin] PATH_INFO: `/v3/projects/d272b3bb14e745cf978b222c4a756a0e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.079857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4299bf-de8d-4761-88c8-93bacd0e5f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d272b3bb14e745cf978b222c4a756a0e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:44.082020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4299bf-de8d-4761-88c8-93bacd0e5f10 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:44.147771 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7c4299bf-de8d-4761-88c8-93bacd0e5f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd272b3bb14e745cf978b222c4a756a0e', 'request_id': 'req-7c4299bf-de8d-4761-88c8-93bacd0e5f10'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:44.154084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7c4299bf-de8d-4761-88c8-93bacd0e5f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd272b3bb14e745cf978b222c4a756a0e', 'request_id': 'req-7c4299bf-de8d-4761-88c8-93bacd0e5f10'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:44.159923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7c4299bf-de8d-4761-88c8-93bacd0e5f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd272b3bb14e745cf978b222c4a756a0e', 'request_id': 'req-7c4299bf-de8d-4761-88c8-93bacd0e5f10'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:44.173608 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7c4299bf-de8d-4761-88c8-93bacd0e5f10 admin admin] The token cache is being invalidate because project d272b3bb14e745cf978b222c4a756a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:44.173931 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7c4299bf-de8d-4761-88c8-93bacd0e5f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:44.174570 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7c4299bf-de8d-4761-88c8-93bacd0e5f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:44.176229 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1333/2648] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:43 2022] DELETE /identity/v3/projects/d272b3bb14e745cf978b222c4a756a0e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:44.186236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162aed0-4d8e-4b71-aa76-b72c944f450e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.186527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162aed0-4d8e-4b71-aa76-b72c944f450e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.186809 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162aed0-4d8e-4b71-aa76-b72c944f450e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.204795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c162aed0-4d8e-4b71-aa76-b72c944f450e None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:44.206043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c162aed0-4d8e-4b71-aa76-b72c944f450e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:44.272733 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1316/2649] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:44.279059 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a386374-b96c-4ce8-bf65-eb324a5a29d6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.279400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a386374-b96c-4ce8-bf65-eb324a5a29d6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.279748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a386374-b96c-4ce8-bf65-eb324a5a29d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.302163 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2a386374-b96c-4ce8-bf65-eb324a5a29d6 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:44.303411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2a386374-b96c-4ce8-bf65-eb324a5a29d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:44.319902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1334/2650] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:44.324952 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162aed0-4d8e-4b71-aa76-b72c944f450e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:44.335709 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162aed0-4d8e-4b71-aa76-b72c944f450e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:44.340396 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177ba06-5062-466b-9ded-795e46f6040a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:44.341124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177ba06-5062-466b-9ded-795e46f6040a admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.341409 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177ba06-5062-466b-9ded-795e46f6040a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.341687 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177ba06-5062-466b-9ded-795e46f6040a admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.342455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177ba06-5062-466b-9ded-795e46f6040a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:44.342763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177ba06-5062-466b-9ded-795e46f6040a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:44.344462 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177ba06-5062-466b-9ded-795e46f6040a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:44.358132 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1317/2651] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:24:44.362766 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a386374-b96c-4ce8-bf65-eb324a5a29d6 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:44.372906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a386374-b96c-4ce8-bf65-eb324a5a29d6 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:44.377662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5eee23-1bcc-44eb-8bd0-cbebda558836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:44.378366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5eee23-1bcc-44eb-8bd0-cbebda558836 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.378647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5eee23-1bcc-44eb-8bd0-cbebda558836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.378921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5eee23-1bcc-44eb-8bd0-cbebda558836 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.379625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5eee23-1bcc-44eb-8bd0-cbebda558836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:44.381625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5eee23-1bcc-44eb-8bd0-cbebda558836 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:44.409234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1335/2652] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:44 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:44.413420 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177ba06-5062-466b-9ded-795e46f6040a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:44.424480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177ba06-5062-466b-9ded-795e46f6040a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:44.491336 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b1a2a-2e23-4fc4-b954-5aaee76543e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:44.492419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b1a2a-2e23-4fc4-b954-5aaee76543e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.492876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b1a2a-2e23-4fc4-b954-5aaee76543e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.493329 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b1a2a-2e23-4fc4-b954-5aaee76543e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.494695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b1a2a-2e23-4fc4-b954-5aaee76543e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:44.497475 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b1a2a-2e23-4fc4-b954-5aaee76543e3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:44.518228 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1318/2653] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:44 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:44.522431 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5eee23-1bcc-44eb-8bd0-cbebda558836 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:44.533709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5eee23-1bcc-44eb-8bd0-cbebda558836 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:44.538542 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49de91-675c-4efd-9e0e-783c2a52f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:44.539253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49de91-675c-4efd-9e0e-783c2a52f1ad admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.539534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49de91-675c-4efd-9e0e-783c2a52f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.539902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49de91-675c-4efd-9e0e-783c2a52f1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.540820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49de91-675c-4efd-9e0e-783c2a52f1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:44.542622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49de91-675c-4efd-9e0e-783c2a52f1ad admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:44.554903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1336/2654] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:44.558946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b1a2a-2e23-4fc4-b954-5aaee76543e3 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:44.569270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b1a2a-2e23-4fc4-b954-5aaee76543e3 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:44.574238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e748f4-9606-4fe6-82b9-8187881d5983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:44.574899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e748f4-9606-4fe6-82b9-8187881d5983 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.575505 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e748f4-9606-4fe6-82b9-8187881d5983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.575798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e748f4-9606-4fe6-82b9-8187881d5983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.576527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e748f4-9606-4fe6-82b9-8187881d5983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:44.578174 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e748f4-9606-4fe6-82b9-8187881d5983 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:44.589798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1319/2655] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:44.594564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49de91-675c-4efd-9e0e-783c2a52f1ad admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:44.605742 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49de91-675c-4efd-9e0e-783c2a52f1ad admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:44.611264 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98badd-d94a-4795-80be-966125f57033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:44.612159 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98badd-d94a-4795-80be-966125f57033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.612463 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98badd-d94a-4795-80be-966125f57033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.612740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98badd-d94a-4795-80be-966125f57033 admin admin] PATH_INFO: `/v3/projects/d066637aa68645d8b135dbac8a557927/users/3caf4b203cb54d6a9e871493c0e8cee6/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.628102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca98badd-d94a-4795-80be-966125f57033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d066637aa68645d8b135dbac8a557927, user_id=3caf4b203cb54d6a9e871493c0e8cee6, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:44.630909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca98badd-d94a-4795-80be-966125f57033 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:44.644912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1337/2656] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:44 2022] PUT /identity/v3/projects/d066637aa68645d8b135dbac8a557927/users/3caf4b203cb54d6a9e871493c0e8cee6/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:44.656113 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2925f-7ce8-4356-bba6-24e063ba8692 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.656395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2925f-7ce8-4356-bba6-24e063ba8692 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.656698 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2925f-7ce8-4356-bba6-24e063ba8692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.689057 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-aed2925f-7ce8-4356-bba6-24e063ba8692 None None] MFA Rules not processed for user `3caf4b203cb54d6a9e871493c0e8cee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:44.690216 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-aed2925f-7ce8-4356-bba6-24e063ba8692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:44.702153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:44.759422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1320/2657] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:44.763980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98badd-d94a-4795-80be-966125f57033 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:44.799241 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:44.831853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98badd-d94a-4795-80be-966125f57033 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:44.836660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ea31b-c0b5-44a5-9ccc-6818078f9653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:44.837315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ea31b-c0b5-44a5-9ccc-6818078f9653 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.837592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ea31b-c0b5-44a5-9ccc-6818078f9653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.837898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ea31b-c0b5-44a5-9ccc-6818078f9653 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.838586 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ea31b-c0b5-44a5-9ccc-6818078f9653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:44.840526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ea31b-c0b5-44a5-9ccc-6818078f9653 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:44.867370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1338/2658] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:44 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:24:44.871640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2925f-7ce8-4356-bba6-24e063ba8692 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:44.882532 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2925f-7ce8-4356-bba6-24e063ba8692 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:44.946482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab37770-067e-48b3-bd74-0ed6175ae354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:44.947078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab37770-067e-48b3-bd74-0ed6175ae354 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.947357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab37770-067e-48b3-bd74-0ed6175ae354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.947627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab37770-067e-48b3-bd74-0ed6175ae354 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.948315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab37770-067e-48b3-bd74-0ed6175ae354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:44.950808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab37770-067e-48b3-bd74-0ed6175ae354 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:44.975208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1321/2659] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:44 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:44.979363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ea31b-c0b5-44a5-9ccc-6818078f9653 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:44.989906 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ea31b-c0b5-44a5-9ccc-6818078f9653 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:44.994567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853675a7-8189-40c8-a13c-3837fb63b0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:44.995227 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853675a7-8189-40c8-a13c-3837fb63b0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:44.995505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853675a7-8189-40c8-a13c-3837fb63b0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:44.995772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853675a7-8189-40c8-a13c-3837fb63b0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:44.996488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853675a7-8189-40c8-a13c-3837fb63b0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:44.998969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853675a7-8189-40c8-a13c-3837fb63b0e8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:45.012484 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1339/2660] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:45.017335 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab37770-067e-48b3-bd74-0ed6175ae354 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:45.027474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab37770-067e-48b3-bd74-0ed6175ae354 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:45.033095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7478ea6-c87e-4270-b1a6-ab724da9e732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:45.034002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7478ea6-c87e-4270-b1a6-ab724da9e732 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:45.034283 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7478ea6-c87e-4270-b1a6-ab724da9e732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:45.034555 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7478ea6-c87e-4270-b1a6-ab724da9e732 admin admin] PATH_INFO: `/v3/projects/209378a5b68847af859b8acd52f632a5/users/a2d6280035bc42e0a1f4e6311794f0b6/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:45.049414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7478ea6-c87e-4270-b1a6-ab724da9e732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209378a5b68847af859b8acd52f632a5, user_id=a2d6280035bc42e0a1f4e6311794f0b6, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:45.052413 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7478ea6-c87e-4270-b1a6-ab724da9e732 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:45.066701 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1322/2661] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:45 2022] PUT /identity/v3/projects/209378a5b68847af859b8acd52f632a5/users/a2d6280035bc42e0a1f4e6311794f0b6/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:45.070965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853675a7-8189-40c8-a13c-3837fb63b0e8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:45.082373 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853675a7-8189-40c8-a13c-3837fb63b0e8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:45.152001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571aecc2-435f-45bc-96a0-e62dbffc8ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:45.152702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571aecc2-435f-45bc-96a0-e62dbffc8ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:45.152993 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571aecc2-435f-45bc-96a0-e62dbffc8ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:45.153254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571aecc2-435f-45bc-96a0-e62dbffc8ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:45.155899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571aecc2-435f-45bc-96a0-e62dbffc8ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:45.157530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571aecc2-435f-45bc-96a0-e62dbffc8ed2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:45.172839 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1340/2662] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:45.177207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7478ea6-c87e-4270-b1a6-ab724da9e732 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:45.191740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7478ea6-c87e-4270-b1a6-ab724da9e732 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:45.196814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87103bc-69bc-4949-b60b-325a6d64d7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:45.197638 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87103bc-69bc-4949-b60b-325a6d64d7d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:45.197963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87103bc-69bc-4949-b60b-325a6d64d7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:45.198233 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87103bc-69bc-4949-b60b-325a6d64d7d7 admin admin] PATH_INFO: `/v3/projects/209378a5b68847af859b8acd52f632a5/users/a2d6280035bc42e0a1f4e6311794f0b6/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:45.203757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87103bc-69bc-4949-b60b-325a6d64d7d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209378a5b68847af859b8acd52f632a5, user_id=a2d6280035bc42e0a1f4e6311794f0b6, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:45.208988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87103bc-69bc-4949-b60b-325a6d64d7d7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:45.222399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1323/2663] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:45 2022] PUT /identity/v3/projects/209378a5b68847af859b8acd52f632a5/users/a2d6280035bc42e0a1f4e6311794f0b6/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:45.235403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656481d-16c0-41eb-88bc-cc39fefdad6d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:45.235891 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656481d-16c0-41eb-88bc-cc39fefdad6d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:45.236739 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656481d-16c0-41eb-88bc-cc39fefdad6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:45.265219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e656481d-16c0-41eb-88bc-cc39fefdad6d None None] MFA Rules not processed for user `a2d6280035bc42e0a1f4e6311794f0b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:45.266492 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e656481d-16c0-41eb-88bc-cc39fefdad6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:45.365933 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1341/2664] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:45.377744 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87103bc-69bc-4949-b60b-325a6d64d7d7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:45.379520 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e87103bc-69bc-4949-b60b-325a6d64d7d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:45.449784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87103bc-69bc-4949-b60b-325a6d64d7d7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:45.454122 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b542a-92d5-4f83-95ad-bd7abb88a226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:45.454763 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b542a-92d5-4f83-95ad-bd7abb88a226 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:45.455048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b542a-92d5-4f83-95ad-bd7abb88a226 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:45.455323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b542a-92d5-4f83-95ad-bd7abb88a226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:45.464652 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b542a-92d5-4f83-95ad-bd7abb88a226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:45.466429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b542a-92d5-4f83-95ad-bd7abb88a226 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:45.475794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1324/2665] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:45.571315 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656481d-16c0-41eb-88bc-cc39fefdad6d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:45.581317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656481d-16c0-41eb-88bc-cc39fefdad6d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:45.585171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c930249-f1da-40f8-ab8a-9e92df512110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:45.585887 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c930249-f1da-40f8-ab8a-9e92df512110 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:45.586225 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c930249-f1da-40f8-ab8a-9e92df512110 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:45.586575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c930249-f1da-40f8-ab8a-9e92df512110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:45.613387 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:45.648274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c930249-f1da-40f8-ab8a-9e92df512110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:45.649767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c930249-f1da-40f8-ab8a-9e92df512110 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:45.658967 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1342/2666] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:50.594778 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b542a-92d5-4f83-95ad-bd7abb88a226 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:50.604835 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:50.663481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b542a-92d5-4f83-95ad-bd7abb88a226 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:50.674441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:50.675357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:50.675638 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:50.675907 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] PATH_INFO: `/v3/users/3caf4b203cb54d6a9e871493c0e8cee6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:50.677418 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3caf4b203cb54d6a9e871493c0e8cee6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:50.679441 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:50.731002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3caf4b203cb54d6a9e871493c0e8cee6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:50.742104 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3caf4b203cb54d6a9e871493c0e8cee6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:50.756092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3caf4b203cb54d6a9e871493c0e8cee6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:50.756783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3caf4b203cb54d6a9e871493c0e8cee6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:50.764548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3caf4b203cb54d6a9e871493c0e8cee6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:50.771779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1325/2667] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:50 2022] DELETE /identity/v3/users/3caf4b203cb54d6a9e871493c0e8cee6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:50.777087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39635353-cf32-4adb-af71-00581f028a2d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:50.777537 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39635353-cf32-4adb-af71-00581f028a2d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:50.777690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39635353-cf32-4adb-af71-00581f028a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:50.795586 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-39635353-cf32-4adb-af71-00581f028a2d None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:50.796779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-39635353-cf32-4adb-af71-00581f028a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:50.853924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1343/2668] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:50.864173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:50.865601 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:50.926989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9832180-a8ce-4850-87ba-b8a43d1778b1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:50.931231 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d721fb1-ef5e-4690-ad66-933da2e3c9f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:50.931609 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d721fb1-ef5e-4690-ad66-933da2e3c9f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:50.931885 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d721fb1-ef5e-4690-ad66-933da2e3c9f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:50.932162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d721fb1-ef5e-4690-ad66-933da2e3c9f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:50.943494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d721fb1-ef5e-4690-ad66-933da2e3c9f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:50.944857 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d721fb1-ef5e-4690-ad66-933da2e3c9f0 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:50.956146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1326/2669] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:51.222311 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39635353-cf32-4adb-af71-00581f028a2d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:51.223850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-39635353-cf32-4adb-af71-00581f028a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:51.238117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39635353-cf32-4adb-af71-00581f028a2d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:51.243097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:51.244076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:51.244418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:51.244759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] PATH_INFO: `/v3/projects/d066637aa68645d8b135dbac8a557927` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:51.246074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] RBAC: Authorizing `identity:delete_project(project_id=d066637aa68645d8b135dbac8a557927)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:51.248212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:51.297216 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd066637aa68645d8b135dbac8a557927', 'request_id': 'req-3e6ffd47-2654-425e-bae6-df6d1eafcdab'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.303634 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd066637aa68645d8b135dbac8a557927', 'request_id': 'req-3e6ffd47-2654-425e-bae6-df6d1eafcdab'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.309303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd066637aa68645d8b135dbac8a557927', 'request_id': 'req-3e6ffd47-2654-425e-bae6-df6d1eafcdab'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.322421 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] The token cache is being invalidate because project d066637aa68645d8b135dbac8a557927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:51.322704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.323311 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.324838 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1344/2670] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:51 2022] DELETE /identity/v3/projects/d066637aa68645d8b135dbac8a557927 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:51.328758 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d721fb1-ef5e-4690-ad66-933da2e3c9f0 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:51.338391 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d721fb1-ef5e-4690-ad66-933da2e3c9f0 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:51.368653 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:51.397350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c8ea5-86ff-4bc6-87d5-3eb1be28f60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:51.398242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c8ea5-86ff-4bc6-87d5-3eb1be28f60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:51.398527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c8ea5-86ff-4bc6-87d5-3eb1be28f60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:51.398798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c8ea5-86ff-4bc6-87d5-3eb1be28f60b admin admin] PATH_INFO: `/v3/users/a2d6280035bc42e0a1f4e6311794f0b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:51.399972 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c8ea5-86ff-4bc6-87d5-3eb1be28f60b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2d6280035bc42e0a1f4e6311794f0b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:51.401950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c8ea5-86ff-4bc6-87d5-3eb1be28f60b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:51.449903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-376c8ea5-86ff-4bc6-87d5-3eb1be28f60b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2d6280035bc42e0a1f4e6311794f0b6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.458058 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-376c8ea5-86ff-4bc6-87d5-3eb1be28f60b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2d6280035bc42e0a1f4e6311794f0b6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.471586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-376c8ea5-86ff-4bc6-87d5-3eb1be28f60b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2d6280035bc42e0a1f4e6311794f0b6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.472236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-376c8ea5-86ff-4bc6-87d5-3eb1be28f60b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2d6280035bc42e0a1f4e6311794f0b6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.479498 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-376c8ea5-86ff-4bc6-87d5-3eb1be28f60b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2d6280035bc42e0a1f4e6311794f0b6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.486640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1327/2671] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:51 2022] DELETE /identity/v3/users/a2d6280035bc42e0a1f4e6311794f0b6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:51.772569 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:51.774131 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:51.790187 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ffd47-2654-425e-bae6-df6d1eafcdab admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:51.848199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:51.862633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035f20f-bf98-43bb-a907-065605d5180e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:51.863493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035f20f-bf98-43bb-a907-065605d5180e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:51.863776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035f20f-bf98-43bb-a907-065605d5180e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:51.864051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035f20f-bf98-43bb-a907-065605d5180e admin admin] PATH_INFO: `/v3/projects/209378a5b68847af859b8acd52f632a5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:51.865245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035f20f-bf98-43bb-a907-065605d5180e admin admin] RBAC: Authorizing `identity:delete_project(project_id=209378a5b68847af859b8acd52f632a5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:51.867207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035f20f-bf98-43bb-a907-065605d5180e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:51.923865 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f035f20f-bf98-43bb-a907-065605d5180e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209378a5b68847af859b8acd52f632a5', 'request_id': 'req-f035f20f-bf98-43bb-a907-065605d5180e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.931833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f035f20f-bf98-43bb-a907-065605d5180e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209378a5b68847af859b8acd52f632a5', 'request_id': 'req-f035f20f-bf98-43bb-a907-065605d5180e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.941846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f035f20f-bf98-43bb-a907-065605d5180e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209378a5b68847af859b8acd52f632a5', 'request_id': 'req-f035f20f-bf98-43bb-a907-065605d5180e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.956633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f035f20f-bf98-43bb-a907-065605d5180e admin admin] The token cache is being invalidate because project 209378a5b68847af859b8acd52f632a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:51.956924 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f035f20f-bf98-43bb-a907-065605d5180e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.958367 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f035f20f-bf98-43bb-a907-065605d5180e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:51.959946 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1345/2672] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:51 2022] DELETE /identity/v3/projects/209378a5b68847af859b8acd52f632a5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:51.964926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867a43a-d85d-4a8e-ad9d-17555df25790 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:51.965423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867a43a-d85d-4a8e-ad9d-17555df25790 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:51.965794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867a43a-d85d-4a8e-ad9d-17555df25790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:51.983876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2867a43a-d85d-4a8e-ad9d-17555df25790 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:51.985135 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2867a43a-d85d-4a8e-ad9d-17555df25790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:52.045892 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1328/2673] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:52.055309 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f957c-ac22-4efa-a96e-9d9915d11485 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:52.055611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f957c-ac22-4efa-a96e-9d9915d11485 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:52.055878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f957c-ac22-4efa-a96e-9d9915d11485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:52.076453 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d58f957c-ac22-4efa-a96e-9d9915d11485 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:52.077654 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d58f957c-ac22-4efa-a96e-9d9915d11485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:52.086795 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1346/2674] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:52.091118 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867a43a-d85d-4a8e-ad9d-17555df25790 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:52.101628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867a43a-d85d-4a8e-ad9d-17555df25790 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:52.107379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06faf50-533e-4801-b5b6-1222b6d3a047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:52.108018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06faf50-533e-4801-b5b6-1222b6d3a047 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:52.108307 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06faf50-533e-4801-b5b6-1222b6d3a047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:52.108578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06faf50-533e-4801-b5b6-1222b6d3a047 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:52.109317 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06faf50-533e-4801-b5b6-1222b6d3a047 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:52.109621 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06faf50-533e-4801-b5b6-1222b6d3a047 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:52.111270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06faf50-533e-4801-b5b6-1222b6d3a047 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:52.125203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1329/2675] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:24:52.130111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f957c-ac22-4efa-a96e-9d9915d11485 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:52.140917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f957c-ac22-4efa-a96e-9d9915d11485 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:52.145527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b19b2a-0c9f-42b8-9cb5-824640a22fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:52.146306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b19b2a-0c9f-42b8-9cb5-824640a22fbf admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:52.146587 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b19b2a-0c9f-42b8-9cb5-824640a22fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:52.146858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b19b2a-0c9f-42b8-9cb5-824640a22fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:52.147532 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b19b2a-0c9f-42b8-9cb5-824640a22fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:52.149468 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b19b2a-0c9f-42b8-9cb5-824640a22fbf admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:52.175002 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1347/2676] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:52 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:52.179625 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06faf50-533e-4801-b5b6-1222b6d3a047 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:52.190209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06faf50-533e-4801-b5b6-1222b6d3a047 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:52.223003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:52.253981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a6860-1439-4c96-a085-f4f65fc77d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:52.254657 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a6860-1439-4c96-a085-f4f65fc77d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:52.254938 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a6860-1439-4c96-a085-f4f65fc77d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:52.255208 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a6860-1439-4c96-a085-f4f65fc77d73 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:52.255908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a6860-1439-4c96-a085-f4f65fc77d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:52.257912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a6860-1439-4c96-a085-f4f65fc77d73 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:52.280128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1330/2677] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:52 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:52.284600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b19b2a-0c9f-42b8-9cb5-824640a22fbf admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:52.296025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b19b2a-0c9f-42b8-9cb5-824640a22fbf admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:52.300625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db06c57-a5a6-440d-8209-6c5f62965139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:52.301313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db06c57-a5a6-440d-8209-6c5f62965139 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:52.301624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db06c57-a5a6-440d-8209-6c5f62965139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:52.301929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db06c57-a5a6-440d-8209-6c5f62965139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:52.302911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db06c57-a5a6-440d-8209-6c5f62965139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:52.304556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db06c57-a5a6-440d-8209-6c5f62965139 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:52.316025 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1348/2678] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:52.320037 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a6860-1439-4c96-a085-f4f65fc77d73 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:52.330109 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a6860-1439-4c96-a085-f4f65fc77d73 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:52.334693 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2179cf-e888-4c96-b946-6157f3549ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:52.335342 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2179cf-e888-4c96-b946-6157f3549ffd admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:52.335652 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2179cf-e888-4c96-b946-6157f3549ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:52.335927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2179cf-e888-4c96-b946-6157f3549ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:52.336639 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2179cf-e888-4c96-b946-6157f3549ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:52.338287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2179cf-e888-4c96-b946-6157f3549ffd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:52.348450 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1331/2679] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:52.352582 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db06c57-a5a6-440d-8209-6c5f62965139 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:52.363069 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db06c57-a5a6-440d-8209-6c5f62965139 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:52.367645 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d985e2-4f6c-40b4-b67e-f2e9fd27928e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:52.368510 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d985e2-4f6c-40b4-b67e-f2e9fd27928e admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:52.368790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d985e2-4f6c-40b4-b67e-f2e9fd27928e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:52.369062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d985e2-4f6c-40b4-b67e-f2e9fd27928e admin admin] PATH_INFO: `/v3/projects/dc0657ffbe81419f9731552da153808c/users/ec7d3677b8b747f893a9854da5936c3e/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:52.384019 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d985e2-4f6c-40b4-b67e-f2e9fd27928e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc0657ffbe81419f9731552da153808c, user_id=ec7d3677b8b747f893a9854da5936c3e, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:52.386721 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d985e2-4f6c-40b4-b67e-f2e9fd27928e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:52.399782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1349/2680] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:52 2022] PUT /identity/v3/projects/dc0657ffbe81419f9731552da153808c/users/ec7d3677b8b747f893a9854da5936c3e/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:52.410412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649162df-b367-410a-947a-6e66e7d58243 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:52.410686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649162df-b367-410a-947a-6e66e7d58243 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:52.410960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649162df-b367-410a-947a-6e66e7d58243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:52.437982 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-649162df-b367-410a-947a-6e66e7d58243 None None] MFA Rules not processed for user `ec7d3677b8b747f893a9854da5936c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:52.439178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-649162df-b367-410a-947a-6e66e7d58243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:52.504589 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1332/2681] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:52.515295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d985e2-4f6c-40b4-b67e-f2e9fd27928e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:52.517005 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b2d985e2-4f6c-40b4-b67e-f2e9fd27928e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:52.584038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d985e2-4f6c-40b4-b67e-f2e9fd27928e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:52.591673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd98d9f-ae62-4976-aaa6-0d46104c8850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:52.592623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd98d9f-ae62-4976-aaa6-0d46104c8850 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:52.593076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd98d9f-ae62-4976-aaa6-0d46104c8850 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:52.593351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd98d9f-ae62-4976-aaa6-0d46104c8850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:52.603684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd98d9f-ae62-4976-aaa6-0d46104c8850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:52.605158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd98d9f-ae62-4976-aaa6-0d46104c8850 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:52.615078 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1350/2682] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:53.096964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649162df-b367-410a-947a-6e66e7d58243 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:53.098445 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-649162df-b367-410a-947a-6e66e7d58243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:53.158380 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649162df-b367-410a-947a-6e66e7d58243 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:53.162270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcca044-5bcd-4a52-970d-85aa960a1966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:53.163014 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcca044-5bcd-4a52-970d-85aa960a1966 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:53.163349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcca044-5bcd-4a52-970d-85aa960a1966 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:53.163626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcca044-5bcd-4a52-970d-85aa960a1966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:53.165804 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8bcca044-5bcd-4a52-970d-85aa960a1966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:53.227703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:53.230331 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcca044-5bcd-4a52-970d-85aa960a1966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:53.232545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcca044-5bcd-4a52-970d-85aa960a1966 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:53.243388 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1333/2683] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:53.972710 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd98d9f-ae62-4976-aaa6-0d46104c8850 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:53.974308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1dd98d9f-ae62-4976-aaa6-0d46104c8850 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:53.990045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd98d9f-ae62-4976-aaa6-0d46104c8850 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:53.994910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:53.995905 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:53.996290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:53.996663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] PATH_INFO: `/v3/users/a8a51acb4036485380d2850b028aefcb` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:53.999124 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a51acb4036485380d2850b028aefcb)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:54.001331 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:54.017643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:54.066070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a51acb4036485380d2850b028aefcb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:54.075339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a51acb4036485380d2850b028aefcb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:54.076652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a51acb4036485380d2850b028aefcb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:54.077238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a51acb4036485380d2850b028aefcb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:54.092154 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a51acb4036485380d2850b028aefcb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:54.106087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1351/2684] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:53 2022] DELETE /identity/v3/users/a8a51acb4036485380d2850b028aefcb => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:54.111384 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a4783-be28-4d6c-aafd-ed8315d9aaaf None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:54.111847 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a4783-be28-4d6c-aafd-ed8315d9aaaf None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:54.111963 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a4783-be28-4d6c-aafd-ed8315d9aaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:54.132545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4b3a4783-be28-4d6c-aafd-ed8315d9aaaf None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:54.133838 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4b3a4783-be28-4d6c-aafd-ed8315d9aaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:54.205641 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1334/2685] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:54.219043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:54.221057 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:54.317827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84f2b7-4250-4b31-b43a-16fd05e773f4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:54.326430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea0776-0c55-454d-b94f-aaabbe012983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:54.327255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea0776-0c55-454d-b94f-aaabbe012983 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:54.327550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea0776-0c55-454d-b94f-aaabbe012983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:54.327863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea0776-0c55-454d-b94f-aaabbe012983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:54.338534 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea0776-0c55-454d-b94f-aaabbe012983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:54.340231 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea0776-0c55-454d-b94f-aaabbe012983 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:54.353947 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1352/2686] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:54.704731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a4783-be28-4d6c-aafd-ed8315d9aaaf None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:54.709339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4b3a4783-be28-4d6c-aafd-ed8315d9aaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:54.730580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a4783-be28-4d6c-aafd-ed8315d9aaaf None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:54.735306 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0b251-af59-4155-a6d1-166c2dffb759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:54.736178 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0b251-af59-4155-a6d1-166c2dffb759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:54.736454 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0b251-af59-4155-a6d1-166c2dffb759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:54.736725 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0b251-af59-4155-a6d1-166c2dffb759 admin admin] PATH_INFO: `/v3/projects/2ec551d1b501453a918e9603521cf59b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:54.738016 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0b251-af59-4155-a6d1-166c2dffb759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ec551d1b501453a918e9603521cf59b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:54.740950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0b251-af59-4155-a6d1-166c2dffb759 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:54.806339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fee0b251-af59-4155-a6d1-166c2dffb759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ec551d1b501453a918e9603521cf59b', 'request_id': 'req-fee0b251-af59-4155-a6d1-166c2dffb759'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:54.819416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fee0b251-af59-4155-a6d1-166c2dffb759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ec551d1b501453a918e9603521cf59b', 'request_id': 'req-fee0b251-af59-4155-a6d1-166c2dffb759'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:54.829015 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fee0b251-af59-4155-a6d1-166c2dffb759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ec551d1b501453a918e9603521cf59b', 'request_id': 'req-fee0b251-af59-4155-a6d1-166c2dffb759'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:54.845376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fee0b251-af59-4155-a6d1-166c2dffb759 admin admin] The token cache is being invalidate because project 2ec551d1b501453a918e9603521cf59b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:54.845533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fee0b251-af59-4155-a6d1-166c2dffb759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:54.846335 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fee0b251-af59-4155-a6d1-166c2dffb759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:54.848135 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1335/2687] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:54 2022] DELETE /identity/v3/projects/2ec551d1b501453a918e9603521cf59b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:54.853183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e957e7-1562-4ee4-9189-484085265276 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:54.853474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e957e7-1562-4ee4-9189-484085265276 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:54.853778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e957e7-1562-4ee4-9189-484085265276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:54.878397 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-26e957e7-1562-4ee4-9189-484085265276 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:54.878643 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-26e957e7-1562-4ee4-9189-484085265276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:54.965910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1353/2688] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:54.970133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6982fb-f895-45f2-95b3-a29209a5b003 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:54.970421 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6982fb-f895-45f2-95b3-a29209a5b003 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:54.970715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6982fb-f895-45f2-95b3-a29209a5b003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:54.991574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ee6982fb-f895-45f2-95b3-a29209a5b003 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:54.992783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ee6982fb-f895-45f2-95b3-a29209a5b003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:55.011398 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1336/2689] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:55.013225 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e957e7-1562-4ee4-9189-484085265276 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:55.027749 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e957e7-1562-4ee4-9189-484085265276 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:55.035976 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df623033-72aa-4879-ab9d-b22cda06374a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:55.037028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df623033-72aa-4879-ab9d-b22cda06374a admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:55.037496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df623033-72aa-4879-ab9d-b22cda06374a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:55.037972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df623033-72aa-4879-ab9d-b22cda06374a admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:55.039226 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df623033-72aa-4879-ab9d-b22cda06374a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:55.039847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df623033-72aa-4879-ab9d-b22cda06374a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:55.041700 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df623033-72aa-4879-ab9d-b22cda06374a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:55.062189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1354/2690] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:24:55.066712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6982fb-f895-45f2-95b3-a29209a5b003 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:55.077944 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6982fb-f895-45f2-95b3-a29209a5b003 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:55.082855 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d1300-2c2f-49bb-99b0-79effa2d2569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:55.083527 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d1300-2c2f-49bb-99b0-79effa2d2569 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:55.083811 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d1300-2c2f-49bb-99b0-79effa2d2569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:55.084086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d1300-2c2f-49bb-99b0-79effa2d2569 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:55.084791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d1300-2c2f-49bb-99b0-79effa2d2569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:55.089431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d1300-2c2f-49bb-99b0-79effa2d2569 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:55.127188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1337/2691] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:55 2022] POST /identity/v3/projects => generated 361 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:24:55.128430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df623033-72aa-4879-ab9d-b22cda06374a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:55.139965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df623033-72aa-4879-ab9d-b22cda06374a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:55.161359 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:55.213351 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2792b-1497-494f-a72d-c04424d5d0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:55.214234 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2792b-1497-494f-a72d-c04424d5d0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:55.214597 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2792b-1497-494f-a72d-c04424d5d0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:55.214781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2792b-1497-494f-a72d-c04424d5d0f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:55.215477 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2792b-1497-494f-a72d-c04424d5d0f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:55.217464 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2792b-1497-494f-a72d-c04424d5d0f0 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:55.245304 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1355/2692] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:55 2022] POST /identity/v3/users => generated 281 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:55.249788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d1300-2c2f-49bb-99b0-79effa2d2569 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:55.265173 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d1300-2c2f-49bb-99b0-79effa2d2569 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:55.279584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414e999-becf-49d2-a037-ba32f032ff07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:55.280569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414e999-becf-49d2-a037-ba32f032ff07 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:55.280995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414e999-becf-49d2-a037-ba32f032ff07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:55.281419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414e999-becf-49d2-a037-ba32f032ff07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:55.282332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414e999-becf-49d2-a037-ba32f032ff07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:55.284194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414e999-becf-49d2-a037-ba32f032ff07 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:55.298446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1338/2693] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:55.302684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2792b-1497-494f-a72d-c04424d5d0f0 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:55.317998 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2792b-1497-494f-a72d-c04424d5d0f0 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:55.326644 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18c111-5e1c-4b75-990a-bed5de63d52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:55.327407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18c111-5e1c-4b75-990a-bed5de63d52c admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:55.328628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18c111-5e1c-4b75-990a-bed5de63d52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:55.328628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18c111-5e1c-4b75-990a-bed5de63d52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:55.328847 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18c111-5e1c-4b75-990a-bed5de63d52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:55.330408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18c111-5e1c-4b75-990a-bed5de63d52c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:55.341957 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1356/2694] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:55.346438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414e999-becf-49d2-a037-ba32f032ff07 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:55.357717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414e999-becf-49d2-a037-ba32f032ff07 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:55.364382 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9146e5b-dc5b-42d6-9435-5748259d39a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:55.365410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9146e5b-dc5b-42d6-9435-5748259d39a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:55.365801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9146e5b-dc5b-42d6-9435-5748259d39a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:55.366151 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9146e5b-dc5b-42d6-9435-5748259d39a2 admin admin] PATH_INFO: `/v3/projects/5f55ae88f9a8449cbe2f7eb61b268da9/users/39e11e99b9454472868a360a493c256b/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:55.385341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9146e5b-dc5b-42d6-9435-5748259d39a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f55ae88f9a8449cbe2f7eb61b268da9, user_id=39e11e99b9454472868a360a493c256b, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:55.388398 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9146e5b-dc5b-42d6-9435-5748259d39a2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:55.405678 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1339/2695] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:55 2022] PUT /identity/v3/projects/5f55ae88f9a8449cbe2f7eb61b268da9/users/39e11e99b9454472868a360a493c256b/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:55.415960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12be8-c852-4519-b275-131da6b71c9f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:55.416537 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12be8-c852-4519-b275-131da6b71c9f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:55.416975 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12be8-c852-4519-b275-131da6b71c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:55.458375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ebf12be8-c852-4519-b275-131da6b71c9f None None] MFA Rules not processed for user `39e11e99b9454472868a360a493c256b`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:55.459730 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ebf12be8-c852-4519-b275-131da6b71c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:55.538292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1357/2696] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:55.576272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9146e5b-dc5b-42d6-9435-5748259d39a2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:55.578125 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b9146e5b-dc5b-42d6-9435-5748259d39a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:55.597681 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:55.643039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9146e5b-dc5b-42d6-9435-5748259d39a2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:55.647716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04893801-f06c-48e1-a7a8-0e6cd5045ccc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:55.648419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04893801-f06c-48e1-a7a8-0e6cd5045ccc service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:55.648705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04893801-f06c-48e1-a7a8-0e6cd5045ccc service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:55.649124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04893801-f06c-48e1-a7a8-0e6cd5045ccc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:55.658880 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04893801-f06c-48e1-a7a8-0e6cd5045ccc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:55.660433 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04893801-f06c-48e1-a7a8-0e6cd5045ccc service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:55.670410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1340/2697] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:24:55.686090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12be8-c852-4519-b275-131da6b71c9f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:55.785206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12be8-c852-4519-b275-131da6b71c9f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:55.790380 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd7546-5d24-4c60-a8b1-91a761ce932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:55.791488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd7546-5d24-4c60-a8b1-91a761ce932b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:55.791827 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd7546-5d24-4c60-a8b1-91a761ce932b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:55.792287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd7546-5d24-4c60-a8b1-91a761ce932b admin admin] PATH_INFO: `/v3/users/39e11e99b9454472868a360a493c256b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:55.793817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd7546-5d24-4c60-a8b1-91a761ce932b admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e11e99b9454472868a360a493c256b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:55.796303 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd7546-5d24-4c60-a8b1-91a761ce932b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:55.852317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:55.878115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-aafd7546-5d24-4c60-a8b1-91a761ce932b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e11e99b9454472868a360a493c256b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:55.897358 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-aafd7546-5d24-4c60-a8b1-91a761ce932b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e11e99b9454472868a360a493c256b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:55.898066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-aafd7546-5d24-4c60-a8b1-91a761ce932b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e11e99b9454472868a360a493c256b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:55.898719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-aafd7546-5d24-4c60-a8b1-91a761ce932b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e11e99b9454472868a360a493c256b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:55.899109 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04893801-f06c-48e1-a7a8-0e6cd5045ccc service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:55.901458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-04893801-f06c-48e1-a7a8-0e6cd5045ccc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:55.913763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-aafd7546-5d24-4c60-a8b1-91a761ce932b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e11e99b9454472868a360a493c256b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:55.924248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04893801-f06c-48e1-a7a8-0e6cd5045ccc service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:55.930488 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18deea-a8ca-4104-a271-7794d8bc246c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:55.931444 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18deea-a8ca-4104-a271-7794d8bc246c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:55.931734 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18deea-a8ca-4104-a271-7794d8bc246c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:55.932037 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18deea-a8ca-4104-a271-7794d8bc246c admin admin] PATH_INFO: `/v3/users/ec7d3677b8b747f893a9854da5936c3e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:55.933484 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18deea-a8ca-4104-a271-7794d8bc246c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec7d3677b8b747f893a9854da5936c3e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:55.935446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18deea-a8ca-4104-a271-7794d8bc246c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:55.935689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1358/2698] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:55 2022] DELETE /identity/v3/users/39e11e99b9454472868a360a493c256b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:55.941106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b810a82-1749-4607-b000-ec6845cc9418 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:55.941306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b810a82-1749-4607-b000-ec6845cc9418 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:55.941574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b810a82-1749-4607-b000-ec6845cc9418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:55.970584 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9b810a82-1749-4607-b000-ec6845cc9418 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:55.971514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9b810a82-1749-4607-b000-ec6845cc9418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:56.028000 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3f18deea-a8ca-4104-a271-7794d8bc246c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec7d3677b8b747f893a9854da5936c3e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:56.048528 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1359/2699] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:56.060475 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3f18deea-a8ca-4104-a271-7794d8bc246c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec7d3677b8b747f893a9854da5936c3e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:56.062133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b810a82-1749-4607-b000-ec6845cc9418 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:56.063843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9b810a82-1749-4607-b000-ec6845cc9418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:56.105249 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3f18deea-a8ca-4104-a271-7794d8bc246c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec7d3677b8b747f893a9854da5936c3e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:56.106904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3f18deea-a8ca-4104-a271-7794d8bc246c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec7d3677b8b747f893a9854da5936c3e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:56.122895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3f18deea-a8ca-4104-a271-7794d8bc246c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec7d3677b8b747f893a9854da5936c3e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:56.144482 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b810a82-1749-4607-b000-ec6845cc9418 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:56.157367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1341/2700] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:55 2022] DELETE /identity/v3/users/ec7d3677b8b747f893a9854da5936c3e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:56.168389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185a672-f215-4614-9d7e-54ace3afcf27 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:56.169224 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185a672-f215-4614-9d7e-54ace3afcf27 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:56.170766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185a672-f215-4614-9d7e-54ace3afcf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:56.207188 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9185a672-f215-4614-9d7e-54ace3afcf27 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:56.208381 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9185a672-f215-4614-9d7e-54ace3afcf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:56.282189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d312d-11ac-450f-8919-2885e460d269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:56.283236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d312d-11ac-450f-8919-2885e460d269 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:56.283876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d312d-11ac-450f-8919-2885e460d269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:56.284283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d312d-11ac-450f-8919-2885e460d269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:56.286606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a55d312d-11ac-450f-8919-2885e460d269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:56.307290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1342/2701] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:56.324381 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185a672-f215-4614-9d7e-54ace3afcf27 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:56.326087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9185a672-f215-4614-9d7e-54ace3afcf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:56.326238 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:56.343613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185a672-f215-4614-9d7e-54ace3afcf27 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:56.349214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e02f2-02fd-43a5-a164-82affffca5ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:56.356923 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e02f2-02fd-43a5-a164-82affffca5ad service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:56.357661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e02f2-02fd-43a5-a164-82affffca5ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:56.358182 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e02f2-02fd-43a5-a164-82affffca5ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:56.374036 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e02f2-02fd-43a5-a164-82affffca5ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:56.377214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e02f2-02fd-43a5-a164-82affffca5ad service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:56.391712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1343/2702] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:24:56.413944 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d312d-11ac-450f-8919-2885e460d269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:56.416611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d312d-11ac-450f-8919-2885e460d269 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:56.441292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1360/2703] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:56.677964 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e02f2-02fd-43a5-a164-82affffca5ad service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:56.679731 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d21e02f2-02fd-43a5-a164-82affffca5ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:56.737483 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:56.818843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e02f2-02fd-43a5-a164-82affffca5ad service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:56.826736 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe6381-adf1-471c-8ae7-fda6fa6f0745 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:56.827769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe6381-adf1-471c-8ae7-fda6fa6f0745 service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:56.828124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe6381-adf1-471c-8ae7-fda6fa6f0745 service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:56.828498 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe6381-adf1-471c-8ae7-fda6fa6f0745 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:56.831724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-edfe6381-adf1-471c-8ae7-fda6fa6f0745 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:56.846304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe6381-adf1-471c-8ae7-fda6fa6f0745 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:56.848139 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe6381-adf1-471c-8ae7-fda6fa6f0745 service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:56.860163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1344/2704] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:24:56.924340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d312d-11ac-450f-8919-2885e460d269 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:56.925996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a55d312d-11ac-450f-8919-2885e460d269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:56.945390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d312d-11ac-450f-8919-2885e460d269 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:56.951449 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd089d3-dc06-4721-a31a-046cddb9b414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:56.952366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd089d3-dc06-4721-a31a-046cddb9b414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:56.952636 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd089d3-dc06-4721-a31a-046cddb9b414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:56.952903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd089d3-dc06-4721-a31a-046cddb9b414 admin admin] PATH_INFO: `/v3/projects/5f55ae88f9a8449cbe2f7eb61b268da9` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:56.954212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd089d3-dc06-4721-a31a-046cddb9b414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f55ae88f9a8449cbe2f7eb61b268da9)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:56.956168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd089d3-dc06-4721-a31a-046cddb9b414 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:57.022999 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dbd089d3-dc06-4721-a31a-046cddb9b414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f55ae88f9a8449cbe2f7eb61b268da9', 'request_id': 'req-dbd089d3-dc06-4721-a31a-046cddb9b414'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.032475 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dbd089d3-dc06-4721-a31a-046cddb9b414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f55ae88f9a8449cbe2f7eb61b268da9', 'request_id': 'req-dbd089d3-dc06-4721-a31a-046cddb9b414'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.040954 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe6381-adf1-471c-8ae7-fda6fa6f0745 service placement] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.042188 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dbd089d3-dc06-4721-a31a-046cddb9b414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f55ae88f9a8449cbe2f7eb61b268da9', 'request_id': 'req-dbd089d3-dc06-4721-a31a-046cddb9b414'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.042973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-edfe6381-adf1-471c-8ae7-fda6fa6f0745 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:57.063489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe6381-adf1-471c-8ae7-fda6fa6f0745 service placement] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:57.063966 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dbd089d3-dc06-4721-a31a-046cddb9b414 admin admin] The token cache is being invalidate because project 5f55ae88f9a8449cbe2f7eb61b268da9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:57.064454 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dbd089d3-dc06-4721-a31a-046cddb9b414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.064958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dbd089d3-dc06-4721-a31a-046cddb9b414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.067683 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1361/2705] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:56 2022] DELETE /identity/v3/projects/5f55ae88f9a8449cbe2f7eb61b268da9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:57.072664 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb1206-56f6-458f-b489-63eb645a9171 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.073779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb1206-56f6-458f-b489-63eb645a9171 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.074560 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb1206-56f6-458f-b489-63eb645a9171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.096092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9ceb1206-56f6-458f-b489-63eb645a9171 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:57.097331 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9ceb1206-56f6-458f-b489-63eb645a9171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:57.142003 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4848e-a230-4836-bc68-8d9572e25197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:57.142889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4848e-a230-4836-bc68-8d9572e25197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.143198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4848e-a230-4836-bc68-8d9572e25197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.143480 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4848e-a230-4836-bc68-8d9572e25197 admin admin] PATH_INFO: `/v3/users/7f2ffbc59c60455cb9cadda081e30381` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.145053 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4848e-a230-4836-bc68-8d9572e25197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2ffbc59c60455cb9cadda081e30381)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:57.147220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4848e-a230-4836-bc68-8d9572e25197 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:57.170604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1362/2706] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:57.175859 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb1206-56f6-458f-b489-63eb645a9171 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.178071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9ceb1206-56f6-458f-b489-63eb645a9171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:57.193933 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb1206-56f6-458f-b489-63eb645a9171 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:57.206224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:57.206224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.206224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.206224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] PATH_INFO: `/v3/projects/dc0657ffbe81419f9731552da153808c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.206711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc0657ffbe81419f9731552da153808c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:57.206711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:57.224202 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-02f4848e-a230-4836-bc68-8d9572e25197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2ffbc59c60455cb9cadda081e30381'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.233678 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-02f4848e-a230-4836-bc68-8d9572e25197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2ffbc59c60455cb9cadda081e30381'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.248066 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-02f4848e-a230-4836-bc68-8d9572e25197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2ffbc59c60455cb9cadda081e30381'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.249128 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-02f4848e-a230-4836-bc68-8d9572e25197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2ffbc59c60455cb9cadda081e30381'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.254090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:57.256739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-02f4848e-a230-4836-bc68-8d9572e25197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2ffbc59c60455cb9cadda081e30381'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.264251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1345/2707] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:57 2022] DELETE /identity/v3/users/7f2ffbc59c60455cb9cadda081e30381 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:57.265358 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c90db-1122-4fe3-b671-fcb760c77252 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.265914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c90db-1122-4fe3-b671-fcb760c77252 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.265914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c90db-1122-4fe3-b671-fcb760c77252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.279271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc0657ffbe81419f9731552da153808c', 'request_id': 'req-47f2c7c3-5d80-455a-9d1f-71fce2002098'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.291012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc0657ffbe81419f9731552da153808c', 'request_id': 'req-47f2c7c3-5d80-455a-9d1f-71fce2002098'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.297545 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7e9c90db-1122-4fe3-b671-fcb760c77252 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:57.298959 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7e9c90db-1122-4fe3-b671-fcb760c77252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:57.301867 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc0657ffbe81419f9731552da153808c', 'request_id': 'req-47f2c7c3-5d80-455a-9d1f-71fce2002098'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.329067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:57.333884 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] The token cache is being invalidate because project dc0657ffbe81419f9731552da153808c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:57.334070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.334788 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.336631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1363/2708] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:57 2022] DELETE /identity/v3/projects/dc0657ffbe81419f9731552da153808c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:57.386479 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1346/2709] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:57.393062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.497273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2c7c3-5d80-455a-9d1f-71fce2002098 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:57.503399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c90db-1122-4fe3-b671-fcb760c77252 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.505367 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47896de7-5156-4348-b774-68f888db704b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:57.506044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7e9c90db-1122-4fe3-b671-fcb760c77252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:57.506210 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47896de7-5156-4348-b774-68f888db704b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.506452 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47896de7-5156-4348-b774-68f888db704b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.506684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47896de7-5156-4348-b774-68f888db704b admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.507524 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47896de7-5156-4348-b774-68f888db704b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:57.507879 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47896de7-5156-4348-b774-68f888db704b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:57.509594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47896de7-5156-4348-b774-68f888db704b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:57.520326 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c90db-1122-4fe3-b671-fcb760c77252 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:57.525595 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:57.526587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.526869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.527143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8 admin admin] PATH_INFO: `/v3/projects/84dc70884aff4a799bb8f38899766e81` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.528781 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84dc70884aff4a799bb8f38899766e81)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:57.531035 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:57.540138 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1364/2710] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:24:57.545529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47896de7-5156-4348-b774-68f888db704b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.562218 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47896de7-5156-4348-b774-68f888db704b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:57.567516 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656f1c6-7cce-4b13-9c74-2df52c1e541d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:57.568289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656f1c6-7cce-4b13-9c74-2df52c1e541d admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.568654 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656f1c6-7cce-4b13-9c74-2df52c1e541d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.569005 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656f1c6-7cce-4b13-9c74-2df52c1e541d admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.569835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656f1c6-7cce-4b13-9c74-2df52c1e541d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:57.571941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656f1c6-7cce-4b13-9c74-2df52c1e541d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:57.605712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1365/2711] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:57 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:24:57.608273 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656f1c6-7cce-4b13-9c74-2df52c1e541d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.609458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84dc70884aff4a799bb8f38899766e81', 'request_id': 'req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.617573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84dc70884aff4a799bb8f38899766e81', 'request_id': 'req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.625666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84dc70884aff4a799bb8f38899766e81', 'request_id': 'req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.639201 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8 admin admin] The token cache is being invalidate because project 84dc70884aff4a799bb8f38899766e81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:24:57.639801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.640785 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-629f66e9-9dd1-41fa-90e4-e2f7be7884e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:57.642631 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1347/2712] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:24:57 2022] DELETE /identity/v3/projects/84dc70884aff4a799bb8f38899766e81 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:57.650356 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf554bb-3982-4ae3-9a66-76466221569b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.650644 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf554bb-3982-4ae3-9a66-76466221569b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.650920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf554bb-3982-4ae3-9a66-76466221569b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.674107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5bf554bb-3982-4ae3-9a66-76466221569b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:57.675469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5bf554bb-3982-4ae3-9a66-76466221569b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:57.681310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656f1c6-7cce-4b13-9c74-2df52c1e541d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:57.684163 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1348/2713] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:57.692290 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec469e-e416-40f6-b7c3-5eb6e60be0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:57.693066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec469e-e416-40f6-b7c3-5eb6e60be0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.693427 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec469e-e416-40f6-b7c3-5eb6e60be0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.693809 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec469e-e416-40f6-b7c3-5eb6e60be0a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.697691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec469e-e416-40f6-b7c3-5eb6e60be0a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:57.697850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95e668-b441-4461-90c7-9584c4c18c03 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.697850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95e668-b441-4461-90c7-9584c4c18c03 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.697850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95e668-b441-4461-90c7-9584c4c18c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.698131 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec469e-e416-40f6-b7c3-5eb6e60be0a7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:57.714587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ba95e668-b441-4461-90c7-9584c4c18c03 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:57.716280 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ba95e668-b441-4461-90c7-9584c4c18c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:57.721115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1366/2714] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:57 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:24:57.722242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec469e-e416-40f6-b7c3-5eb6e60be0a7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.723922 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b8ec469e-e416-40f6-b7c3-5eb6e60be0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:57.724899 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1349/2715] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:57.729096 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95e668-b441-4461-90c7-9584c4c18c03 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.738072 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec469e-e416-40f6-b7c3-5eb6e60be0a7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:57.739150 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95e668-b441-4461-90c7-9584c4c18c03 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:57.742920 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce26d5-6202-423c-94c0-cf8580bd7164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:57.743594 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce26d5-6202-423c-94c0-cf8580bd7164 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.743814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924aad4-c293-4db0-bf02-ab12b62430a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:57.743988 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce26d5-6202-423c-94c0-cf8580bd7164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.744139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce26d5-6202-423c-94c0-cf8580bd7164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.744472 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924aad4-c293-4db0-bf02-ab12b62430a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.744753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924aad4-c293-4db0-bf02-ab12b62430a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.744905 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce26d5-6202-423c-94c0-cf8580bd7164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:57.745091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924aad4-c293-4db0-bf02-ab12b62430a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.745791 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924aad4-c293-4db0-bf02-ab12b62430a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:24:57.746111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924aad4-c293-4db0-bf02-ab12b62430a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:57.746555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce26d5-6202-423c-94c0-cf8580bd7164 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:57.748429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924aad4-c293-4db0-bf02-ab12b62430a0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:57.758412 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1367/2716] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:57.762651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce26d5-6202-423c-94c0-cf8580bd7164 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.764218 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1350/2717] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:24:57.768401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924aad4-c293-4db0-bf02-ab12b62430a0 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.772820 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce26d5-6202-423c-94c0-cf8580bd7164 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:57.777772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cc364-9f03-488e-86fb-26f83a562717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:57.778459 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cc364-9f03-488e-86fb-26f83a562717 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.778744 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cc364-9f03-488e-86fb-26f83a562717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.779219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cc364-9f03-488e-86fb-26f83a562717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.779966 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00cc364-9f03-488e-86fb-26f83a562717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:57.780864 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924aad4-c293-4db0-bf02-ab12b62430a0 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:57.781610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00cc364-9f03-488e-86fb-26f83a562717 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:57.786432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cf525-9894-4b17-8fb2-a6e00b77ca14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:57.787980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cf525-9894-4b17-8fb2-a6e00b77ca14 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.788162 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:57.788368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cf525-9894-4b17-8fb2-a6e00b77ca14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.788368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cf525-9894-4b17-8fb2-a6e00b77ca14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.789566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cf525-9894-4b17-8fb2-a6e00b77ca14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:57.792099 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cf525-9894-4b17-8fb2-a6e00b77ca14 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:57.794613 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1368/2718] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:24:57.798870 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cc364-9f03-488e-86fb-26f83a562717 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.812200 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cc364-9f03-488e-86fb-26f83a562717 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:57.817593 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae074e0-b34d-4534-8680-8a63c57b6131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:57.818527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae074e0-b34d-4534-8680-8a63c57b6131 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.818812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae074e0-b34d-4534-8680-8a63c57b6131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.819088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae074e0-b34d-4534-8680-8a63c57b6131 admin admin] PATH_INFO: `/v3/projects/77a61f18a22c4890b3f8ae6b327a1ca5/users/bdcf4731b9944e79818294f38b17c5bb/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.827386 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1351/2719] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:57 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:24:57.831737 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cf525-9894-4b17-8fb2-a6e00b77ca14 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.841097 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae074e0-b34d-4534-8680-8a63c57b6131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a61f18a22c4890b3f8ae6b327a1ca5, user_id=bdcf4731b9944e79818294f38b17c5bb, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:57.843760 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae074e0-b34d-4534-8680-8a63c57b6131 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:57.858169 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1369/2720] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:57 2022] PUT /identity/v3/projects/77a61f18a22c4890b3f8ae6b327a1ca5/users/bdcf4731b9944e79818294f38b17c5bb/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:57.863253 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1766db-f4bc-47a2-a12c-faceebdc7a89 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.863544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1766db-f4bc-47a2-a12c-faceebdc7a89 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.863821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1766db-f4bc-47a2-a12c-faceebdc7a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.882897 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1c1766db-f4bc-47a2-a12c-faceebdc7a89 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:57.884107 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1c1766db-f4bc-47a2-a12c-faceebdc7a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:57.910214 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cf525-9894-4b17-8fb2-a6e00b77ca14 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:57.915339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:57.950545 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1370/2721] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:57.960514 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1766db-f4bc-47a2-a12c-faceebdc7a89 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:57.962007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1c1766db-f4bc-47a2-a12c-faceebdc7a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:57.987943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785a41a-d82a-4f5b-ac50-79401748360b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:57.988706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785a41a-d82a-4f5b-ac50-79401748360b admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:57.988988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785a41a-d82a-4f5b-ac50-79401748360b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:57.989334 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785a41a-d82a-4f5b-ac50-79401748360b admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:57.990059 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785a41a-d82a-4f5b-ac50-79401748360b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:57.991971 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785a41a-d82a-4f5b-ac50-79401748360b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:58.014794 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1352/2722] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:57 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:24:58.019952 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785a41a-d82a-4f5b-ac50-79401748360b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:58.030795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785a41a-d82a-4f5b-ac50-79401748360b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:58.032623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1766db-f4bc-47a2-a12c-faceebdc7a89 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:58.035647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5b88d-1f5a-4b00-bcd2-b36eb9846865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:58.036323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5b88d-1f5a-4b00-bcd2-b36eb9846865 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.036604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5b88d-1f5a-4b00-bcd2-b36eb9846865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.036776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878b3cf-36d7-41d6-b2c8-cd27b5ea24af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:58.037009 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5b88d-1f5a-4b00-bcd2-b36eb9846865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.037592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5b88d-1f5a-4b00-bcd2-b36eb9846865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:58.037810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878b3cf-36d7-41d6-b2c8-cd27b5ea24af service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.038149 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878b3cf-36d7-41d6-b2c8-cd27b5ea24af service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.038547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878b3cf-36d7-41d6-b2c8-cd27b5ea24af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.039229 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5b88d-1f5a-4b00-bcd2-b36eb9846865 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:58.050900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878b3cf-36d7-41d6-b2c8-cd27b5ea24af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:58.055461 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878b3cf-36d7-41d6-b2c8-cd27b5ea24af service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:58.055661 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1353/2723] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:24:58.057481 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5b88d-1f5a-4b00-bcd2-b36eb9846865 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:58.067467 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1371/2724] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:24:58.078086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5b88d-1f5a-4b00-bcd2-b36eb9846865 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:58.084470 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b08aa0-0b1b-47d7-92f2-3464d838d9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:58.085297 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b08aa0-0b1b-47d7-92f2-3464d838d9ce admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.085685 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b08aa0-0b1b-47d7-92f2-3464d838d9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.086083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b08aa0-0b1b-47d7-92f2-3464d838d9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.086933 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b08aa0-0b1b-47d7-92f2-3464d838d9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:58.088665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b08aa0-0b1b-47d7-92f2-3464d838d9ce admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:58.102646 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1354/2725] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:24:58.107314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878b3cf-36d7-41d6-b2c8-cd27b5ea24af service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:58.128883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878b3cf-36d7-41d6-b2c8-cd27b5ea24af service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:58.141772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205a40f-5b62-461f-b8f1-5af24a387ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:58.143489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205a40f-5b62-461f-b8f1-5af24a387ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.143489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205a40f-5b62-461f-b8f1-5af24a387ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.143709 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205a40f-5b62-461f-b8f1-5af24a387ca3 admin admin] PATH_INFO: `/v3/projects/c3f5e000781b474b9fd24a7ce1e5ab57/users/1388814ad19b4343916ec9a1c1bc67b7/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.167400 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205a40f-5b62-461f-b8f1-5af24a387ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f5e000781b474b9fd24a7ce1e5ab57, user_id=1388814ad19b4343916ec9a1c1bc67b7, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:58.170322 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205a40f-5b62-461f-b8f1-5af24a387ca3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:58.184768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1372/2726] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:58 2022] PUT /identity/v3/projects/c3f5e000781b474b9fd24a7ce1e5ab57/users/1388814ad19b4343916ec9a1c1bc67b7/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:58.195535 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b6044-626a-47c8-86f1-0e5ad95781d4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.195827 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b6044-626a-47c8-86f1-0e5ad95781d4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.196103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b6044-626a-47c8-86f1-0e5ad95781d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.228544 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-db1b6044-626a-47c8-86f1-0e5ad95781d4 None None] MFA Rules not processed for user `1388814ad19b4343916ec9a1c1bc67b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:58.230020 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-db1b6044-626a-47c8-86f1-0e5ad95781d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:58.326654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1355/2727] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:24:58.332460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205a40f-5b62-461f-b8f1-5af24a387ca3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:58.396251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:58.404741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205a40f-5b62-461f-b8f1-5af24a387ca3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:58.415672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1ad80-f78a-4031-927d-73eda8e8c2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:58.416921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1ad80-f78a-4031-927d-73eda8e8c2ec admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.417665 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1ad80-f78a-4031-927d-73eda8e8c2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.418206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1ad80-f78a-4031-927d-73eda8e8c2ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.420832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1ad80-f78a-4031-927d-73eda8e8c2ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:58.426422 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1ad80-f78a-4031-927d-73eda8e8c2ec admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:58.475074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1373/2728] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:24:58 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:24:58.480920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b6044-626a-47c8-86f1-0e5ad95781d4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:58.497690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b6044-626a-47c8-86f1-0e5ad95781d4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:58.578055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85eb243-9286-41f6-9cb3-4cb8df04f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:58.578748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85eb243-9286-41f6-9cb3-4cb8df04f772 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.579238 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85eb243-9286-41f6-9cb3-4cb8df04f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.579719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85eb243-9286-41f6-9cb3-4cb8df04f772 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.580993 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85eb243-9286-41f6-9cb3-4cb8df04f772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:58.583748 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85eb243-9286-41f6-9cb3-4cb8df04f772 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:58.598056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:58.614269 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1356/2729] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:24:58 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:24:58.618695 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1ad80-f78a-4031-927d-73eda8e8c2ec admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:58.632983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1ad80-f78a-4031-927d-73eda8e8c2ec admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:58.639090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902a6de-6dc8-4396-ae3a-02386a890ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:58.639895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902a6de-6dc8-4396-ae3a-02386a890ddf admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.640242 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902a6de-6dc8-4396-ae3a-02386a890ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.640579 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902a6de-6dc8-4396-ae3a-02386a890ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.641425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902a6de-6dc8-4396-ae3a-02386a890ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:58.643287 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902a6de-6dc8-4396-ae3a-02386a890ddf admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:58.656916 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1374/2730] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:24:58.661279 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85eb243-9286-41f6-9cb3-4cb8df04f772 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:58.672626 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85eb243-9286-41f6-9cb3-4cb8df04f772 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:58.678411 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7431d-435c-4b82-a08b-855c01cf02d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:58.679489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7431d-435c-4b82-a08b-855c01cf02d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.679773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7431d-435c-4b82-a08b-855c01cf02d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.680049 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7431d-435c-4b82-a08b-855c01cf02d5 admin admin] PATH_INFO: `/v3/projects/3fca86ba9c4b4bc09e14a35da41829e9/users/a7572547438943db9b7919092d3c881f/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.697321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7431d-435c-4b82-a08b-855c01cf02d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fca86ba9c4b4bc09e14a35da41829e9, user_id=a7572547438943db9b7919092d3c881f, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:58.700219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7431d-435c-4b82-a08b-855c01cf02d5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:58.717094 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1357/2731] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:58 2022] PUT /identity/v3/projects/3fca86ba9c4b4bc09e14a35da41829e9/users/a7572547438943db9b7919092d3c881f/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:58.721280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902a6de-6dc8-4396-ae3a-02386a890ddf admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:58.733606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902a6de-6dc8-4396-ae3a-02386a890ddf admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:58.795757 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cc2d0-ac9a-4615-aa8e-a1ad7f90e29e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.796506 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cc2d0-ac9a-4615-aa8e-a1ad7f90e29e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.796783 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cc2d0-ac9a-4615-aa8e-a1ad7f90e29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.819913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc525ac-45f4-4a02-af8b-e98e13a237a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:58.821062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc525ac-45f4-4a02-af8b-e98e13a237a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.821550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc525ac-45f4-4a02-af8b-e98e13a237a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.822146 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc525ac-45f4-4a02-af8b-e98e13a237a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.823490 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc525ac-45f4-4a02-af8b-e98e13a237a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:58.826237 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2b6cc2d0-ac9a-4615-aa8e-a1ad7f90e29e None None] MFA Rules not processed for user `bdcf4731b9944e79818294f38b17c5bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:58.826681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc525ac-45f4-4a02-af8b-e98e13a237a5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:58.827468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2b6cc2d0-ac9a-4615-aa8e-a1ad7f90e29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:58.841857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1375/2732] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:24:58.843694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc525ac-45f4-4a02-af8b-e98e13a237a5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:58.856209 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc525ac-45f4-4a02-af8b-e98e13a237a5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:58.870935 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d23370-e594-4ef6-930c-a81332b2b091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:58.872525 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d23370-e594-4ef6-930c-a81332b2b091 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.873030 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d23370-e594-4ef6-930c-a81332b2b091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.873523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d23370-e594-4ef6-930c-a81332b2b091 admin admin] PATH_INFO: `/v3/projects/3fca86ba9c4b4bc09e14a35da41829e9/users/a7572547438943db9b7919092d3c881f/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.878029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d23370-e594-4ef6-930c-a81332b2b091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fca86ba9c4b4bc09e14a35da41829e9, user_id=a7572547438943db9b7919092d3c881f, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:58.883060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d23370-e594-4ef6-930c-a81332b2b091 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:58.896132 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1376/2733] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:24:58 2022] PUT /identity/v3/projects/3fca86ba9c4b4bc09e14a35da41829e9/users/a7572547438943db9b7919092d3c881f/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:24:58.908761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27a4a0-8448-430e-a734-6111a62cf2e3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:58.909038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27a4a0-8448-430e-a734-6111a62cf2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:58.909317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27a4a0-8448-430e-a734-6111a62cf2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:58.932290 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1358/2734] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:58.946266 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1e27a4a0-8448-430e-a734-6111a62cf2e3 None None] MFA Rules not processed for user `a7572547438943db9b7919092d3c881f`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:58.948430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1e27a4a0-8448-430e-a734-6111a62cf2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:58.951640 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cc2d0-ac9a-4615-aa8e-a1ad7f90e29e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:58.953927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2b6cc2d0-ac9a-4615-aa8e-a1ad7f90e29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:59.045909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1377/2735] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:59.046349 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cc2d0-ac9a-4615-aa8e-a1ad7f90e29e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:59.058008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27a4a0-8448-430e-a734-6111a62cf2e3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:59.059850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1e27a4a0-8448-430e-a734-6111a62cf2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:59.106743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:59.112101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663348a-d0c9-41f5-aca3-0acfbd912618 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:59.112101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663348a-d0c9-41f5-aca3-0acfbd912618 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:59.112101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663348a-d0c9-41f5-aca3-0acfbd912618 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:59.112101 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663348a-d0c9-41f5-aca3-0acfbd912618 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:59.127189 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27a4a0-8448-430e-a734-6111a62cf2e3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:59.131008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647af92-2055-4372-8690-15d9d31e3b55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:59.131672 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647af92-2055-4372-8690-15d9d31e3b55 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:59.131955 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647af92-2055-4372-8690-15d9d31e3b55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:59.132252 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647af92-2055-4372-8690-15d9d31e3b55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:59.150381 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647af92-2055-4372-8690-15d9d31e3b55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:59.154013 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647af92-2055-4372-8690-15d9d31e3b55 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:59.159251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:24:59.173445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1378/2736] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:59 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:24:59.201194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663348a-d0c9-41f5-aca3-0acfbd912618 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:59.209077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663348a-d0c9-41f5-aca3-0acfbd912618 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:59.219789 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1359/2737] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:24:59.645866 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647af92-2055-4372-8690-15d9d31e3b55 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:59.711804 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647af92-2055-4372-8690-15d9d31e3b55 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:59.717109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:59.718012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:59.718295 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:59.718592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] PATH_INFO: `/v3/users/1388814ad19b4343916ec9a1c1bc67b7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:59.719856 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1388814ad19b4343916ec9a1c1bc67b7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:59.722072 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:24:59.773169 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1388814ad19b4343916ec9a1c1bc67b7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:59.782193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1388814ad19b4343916ec9a1c1bc67b7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:59.782896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1388814ad19b4343916ec9a1c1bc67b7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:59.783496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1388814ad19b4343916ec9a1c1bc67b7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:59.798997 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1388814ad19b4343916ec9a1c1bc67b7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:24:59.817106 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1379/2738] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:24:59 2022] DELETE /identity/v3/users/1388814ad19b4343916ec9a1c1bc67b7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:24:59.821620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649eab7-9a34-4767-98f4-4d7abb096f4b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:59.822203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649eab7-9a34-4767-98f4-4d7abb096f4b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:59.822203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649eab7-9a34-4767-98f4-4d7abb096f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:59.839818 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5649eab7-9a34-4767-98f4-4d7abb096f4b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:24:59.841994 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5649eab7-9a34-4767-98f4-4d7abb096f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:59.910756 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1360/2739] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:24:59.921000 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:24:59.922715 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:24:59.982237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0135f-ffdf-4f33-b5a5-f917102f3561 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:24:59.986371 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2efe2f-8ad6-4858-859a-3cf0dc5faf13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:24:59.987018 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2efe2f-8ad6-4858-859a-3cf0dc5faf13 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:24:59.987327 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2efe2f-8ad6-4858-859a-3cf0dc5faf13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:24:59.987575 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2efe2f-8ad6-4858-859a-3cf0dc5faf13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:24:59.997342 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2efe2f-8ad6-4858-859a-3cf0dc5faf13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:24:59.999120 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2efe2f-8ad6-4858-859a-3cf0dc5faf13 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:00.009413 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1380/2740] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:24:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:25:00.334564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649eab7-9a34-4767-98f4-4d7abb096f4b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:00.336094 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5649eab7-9a34-4767-98f4-4d7abb096f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:00.349486 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649eab7-9a34-4767-98f4-4d7abb096f4b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:00.354892 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e74a2-2cf5-4199-aac4-4075f9ac1042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:00.355762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e74a2-2cf5-4199-aac4-4075f9ac1042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:00.356042 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e74a2-2cf5-4199-aac4-4075f9ac1042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:00.356365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e74a2-2cf5-4199-aac4-4075f9ac1042 admin admin] PATH_INFO: `/v3/projects/c3f5e000781b474b9fd24a7ce1e5ab57` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:00.357557 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e74a2-2cf5-4199-aac4-4075f9ac1042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f5e000781b474b9fd24a7ce1e5ab57)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:00.359541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e74a2-2cf5-4199-aac4-4075f9ac1042 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:00.409973 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-754e74a2-2cf5-4199-aac4-4075f9ac1042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f5e000781b474b9fd24a7ce1e5ab57', 'request_id': 'req-754e74a2-2cf5-4199-aac4-4075f9ac1042'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:00.418332 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-754e74a2-2cf5-4199-aac4-4075f9ac1042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f5e000781b474b9fd24a7ce1e5ab57', 'request_id': 'req-754e74a2-2cf5-4199-aac4-4075f9ac1042'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:00.425670 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-754e74a2-2cf5-4199-aac4-4075f9ac1042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f5e000781b474b9fd24a7ce1e5ab57', 'request_id': 'req-754e74a2-2cf5-4199-aac4-4075f9ac1042'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:00.437587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-754e74a2-2cf5-4199-aac4-4075f9ac1042 admin admin] The token cache is being invalidate because project c3f5e000781b474b9fd24a7ce1e5ab57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:25:00.437822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-754e74a2-2cf5-4199-aac4-4075f9ac1042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:00.438474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-754e74a2-2cf5-4199-aac4-4075f9ac1042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:00.440036 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1361/2741] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:25:00 2022] DELETE /identity/v3/projects/c3f5e000781b474b9fd24a7ce1e5ab57 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:00.444070 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2efe2f-8ad6-4858-859a-3cf0dc5faf13 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:00.449168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:00.456946 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2efe2f-8ad6-4858-859a-3cf0dc5faf13 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:00.517100 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:00.518015 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:00.518301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:00.518578 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] PATH_INFO: `/v3/users/a7572547438943db9b7919092d3c881f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:00.519779 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7572547438943db9b7919092d3c881f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:00.521721 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:00.577883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7572547438943db9b7919092d3c881f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:00.586688 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7572547438943db9b7919092d3c881f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:00.587314 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7572547438943db9b7919092d3c881f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:00.587923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7572547438943db9b7919092d3c881f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:00.601798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7572547438943db9b7919092d3c881f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:00.616462 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1381/2742] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:25:00 2022] DELETE /identity/v3/users/a7572547438943db9b7919092d3c881f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:00.637270 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7898a7a-98a6-42c6-8d53-74c3f4584f23 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:00.637909 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7898a7a-98a6-42c6-8d53-74c3f4584f23 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:00.638423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7898a7a-98a6-42c6-8d53-74c3f4584f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:00.669209 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e7898a7a-98a6-42c6-8d53-74c3f4584f23 None None] MFA Rules not processed for user `bdcf4731b9944e79818294f38b17c5bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:00.671402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e7898a7a-98a6-42c6-8d53-74c3f4584f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:00.742115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:00.760370 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1362/2743] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:01.365556 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:01.367359 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:01.461958 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee56c5f-2854-4cfb-9a51-def97d171f9e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:01.469418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a37e4-5237-4ffb-87c2-b5ba673e60ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:01.470535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a37e4-5237-4ffb-87c2-b5ba673e60ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:01.471090 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a37e4-5237-4ffb-87c2-b5ba673e60ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:01.471446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a37e4-5237-4ffb-87c2-b5ba673e60ac admin admin] PATH_INFO: `/v3/projects/3fca86ba9c4b4bc09e14a35da41829e9` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:01.472711 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a37e4-5237-4ffb-87c2-b5ba673e60ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fca86ba9c4b4bc09e14a35da41829e9)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:01.475171 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a37e4-5237-4ffb-87c2-b5ba673e60ac admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:01.514607 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:01.545626 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-967a37e4-5237-4ffb-87c2-b5ba673e60ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fca86ba9c4b4bc09e14a35da41829e9', 'request_id': 'req-967a37e4-5237-4ffb-87c2-b5ba673e60ac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:01.560266 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-967a37e4-5237-4ffb-87c2-b5ba673e60ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fca86ba9c4b4bc09e14a35da41829e9', 'request_id': 'req-967a37e4-5237-4ffb-87c2-b5ba673e60ac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:01.570597 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-967a37e4-5237-4ffb-87c2-b5ba673e60ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fca86ba9c4b4bc09e14a35da41829e9', 'request_id': 'req-967a37e4-5237-4ffb-87c2-b5ba673e60ac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:01.589799 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-967a37e4-5237-4ffb-87c2-b5ba673e60ac admin admin] The token cache is being invalidate because project 3fca86ba9c4b4bc09e14a35da41829e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:25:01.589799 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-967a37e4-5237-4ffb-87c2-b5ba673e60ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:01.589799 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-967a37e4-5237-4ffb-87c2-b5ba673e60ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:01.592086 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1382/2744] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:25:01 2022] DELETE /identity/v3/projects/3fca86ba9c4b4bc09e14a35da41829e9 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:01.597176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1da017-bfb7-40ce-8c62-1d2e4b1dc2de None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:01.597850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1da017-bfb7-40ce-8c62-1d2e4b1dc2de None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:01.597850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1da017-bfb7-40ce-8c62-1d2e4b1dc2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:01.617387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-8d1da017-bfb7-40ce-8c62-1d2e4b1dc2de None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:01.618568 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8d1da017-bfb7-40ce-8c62-1d2e4b1dc2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:01.690548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1363/2745] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:01.700691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04750b39-dbb4-4b30-ba77-f010f5d47818 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:01.701005 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04750b39-dbb4-4b30-ba77-f010f5d47818 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:01.701311 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04750b39-dbb4-4b30-ba77-f010f5d47818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:01.727211 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-04750b39-dbb4-4b30-ba77-f010f5d47818 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:01.728416 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-04750b39-dbb4-4b30-ba77-f010f5d47818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:01.741050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1383/2746] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:01.742540 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1da017-bfb7-40ce-8c62-1d2e4b1dc2de None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:01.760613 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1da017-bfb7-40ce-8c62-1d2e4b1dc2de None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:01.768134 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e6e14-ac1d-4d72-ba71-75b1de978ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:01.769239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e6e14-ac1d-4d72-ba71-75b1de978ccd admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:01.769464 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e6e14-ac1d-4d72-ba71-75b1de978ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:01.769869 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e6e14-ac1d-4d72-ba71-75b1de978ccd admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:01.770684 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e6e14-ac1d-4d72-ba71-75b1de978ccd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:25:01.773448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e6e14-ac1d-4d72-ba71-75b1de978ccd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:01.773448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e6e14-ac1d-4d72-ba71-75b1de978ccd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:01.787776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1364/2747] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:25:01.792535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04750b39-dbb4-4b30-ba77-f010f5d47818 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:01.810020 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04750b39-dbb4-4b30-ba77-f010f5d47818 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:01.821101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f955b3-e980-4cc5-8386-1fe8ae8b6752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:01.822320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f955b3-e980-4cc5-8386-1fe8ae8b6752 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:01.822774 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f955b3-e980-4cc5-8386-1fe8ae8b6752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:01.823203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f955b3-e980-4cc5-8386-1fe8ae8b6752 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:01.825703 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f955b3-e980-4cc5-8386-1fe8ae8b6752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:01.828800 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f955b3-e980-4cc5-8386-1fe8ae8b6752 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:01.863686 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1384/2748] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:25:01 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:25:01.867712 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e6e14-ac1d-4d72-ba71-75b1de978ccd admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:01.880071 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e6e14-ac1d-4d72-ba71-75b1de978ccd admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:01.950960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d023f-e6fd-4966-afe5-a1047f0c1705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:01.951833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d023f-e6fd-4966-afe5-a1047f0c1705 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:01.952121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d023f-e6fd-4966-afe5-a1047f0c1705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:01.952395 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d023f-e6fd-4966-afe5-a1047f0c1705 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:01.953089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d023f-e6fd-4966-afe5-a1047f0c1705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:01.955113 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d023f-e6fd-4966-afe5-a1047f0c1705 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:01.986351 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1365/2749] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:25:01 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:25:01.990360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f955b3-e980-4cc5-8386-1fe8ae8b6752 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:02.006133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f955b3-e980-4cc5-8386-1fe8ae8b6752 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:02.016567 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c5d5c-bebc-4d0c-a97f-ee93bfca6be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:02.017422 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c5d5c-bebc-4d0c-a97f-ee93bfca6be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:02.017786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c5d5c-bebc-4d0c-a97f-ee93bfca6be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:02.018132 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c5d5c-bebc-4d0c-a97f-ee93bfca6be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:02.018869 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c5d5c-bebc-4d0c-a97f-ee93bfca6be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:02.020577 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c5d5c-bebc-4d0c-a97f-ee93bfca6be9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:02.034729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1385/2750] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:25:02.037719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d023f-e6fd-4966-afe5-a1047f0c1705 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:02.050679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d023f-e6fd-4966-afe5-a1047f0c1705 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:02.056875 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecab8e4-ac69-480e-a532-d08e3927d34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:02.057611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecab8e4-ac69-480e-a532-d08e3927d34d admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:02.057927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecab8e4-ac69-480e-a532-d08e3927d34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:02.058206 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecab8e4-ac69-480e-a532-d08e3927d34d admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:02.058995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecab8e4-ac69-480e-a532-d08e3927d34d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:02.061686 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecab8e4-ac69-480e-a532-d08e3927d34d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:02.073803 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1366/2751] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:25:02.078393 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c5d5c-bebc-4d0c-a97f-ee93bfca6be9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:02.088354 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c5d5c-bebc-4d0c-a97f-ee93bfca6be9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:02.098313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2514b5-9fbe-41b6-b6f8-645a5f830d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:02.098606 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2514b5-9fbe-41b6-b6f8-645a5f830d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:02.098911 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2514b5-9fbe-41b6-b6f8-645a5f830d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:02.099313 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2514b5-9fbe-41b6-b6f8-645a5f830d93 admin admin] PATH_INFO: `/v3/projects/f1c27a26ad3f4d76927193bf22012dd3/users/ae37c581488e4b23ba3196cfdf479e76/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:02.124324 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2514b5-9fbe-41b6-b6f8-645a5f830d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1c27a26ad3f4d76927193bf22012dd3, user_id=ae37c581488e4b23ba3196cfdf479e76, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:02.128250 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2514b5-9fbe-41b6-b6f8-645a5f830d93 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:02.149943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1386/2752] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:25:02 2022] PUT /identity/v3/projects/f1c27a26ad3f4d76927193bf22012dd3/users/ae37c581488e4b23ba3196cfdf479e76/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:02.163469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4563b51-e0de-42e7-b8fc-8f0e0c6b90af None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:02.163846 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4563b51-e0de-42e7-b8fc-8f0e0c6b90af None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:02.164179 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4563b51-e0de-42e7-b8fc-8f0e0c6b90af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:02.196143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c4563b51-e0de-42e7-b8fc-8f0e0c6b90af None None] MFA Rules not processed for user `ae37c581488e4b23ba3196cfdf479e76`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:02.197367 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c4563b51-e0de-42e7-b8fc-8f0e0c6b90af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:02.269018 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:02.288920 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1367/2753] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:02.294384 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2514b5-9fbe-41b6-b6f8-645a5f830d93 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:02.385646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2514b5-9fbe-41b6-b6f8-645a5f830d93 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:02.393805 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524f568-50bc-44e5-86af-cf175e918faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:02.395087 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524f568-50bc-44e5-86af-cf175e918faa admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:02.395624 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524f568-50bc-44e5-86af-cf175e918faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:02.396263 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524f568-50bc-44e5-86af-cf175e918faa admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:02.397489 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524f568-50bc-44e5-86af-cf175e918faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:02.401018 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524f568-50bc-44e5-86af-cf175e918faa admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:02.436206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1387/2754] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:25:02 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:25:02.440376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4563b51-e0de-42e7-b8fc-8f0e0c6b90af None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:02.451751 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4563b51-e0de-42e7-b8fc-8f0e0c6b90af None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:02.533912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab5a45-74d8-4d92-a43b-45dfcc6a9b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:02.534512 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab5a45-74d8-4d92-a43b-45dfcc6a9b93 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:02.534787 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab5a45-74d8-4d92-a43b-45dfcc6a9b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:02.535063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab5a45-74d8-4d92-a43b-45dfcc6a9b93 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:02.535770 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab5a45-74d8-4d92-a43b-45dfcc6a9b93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:02.537710 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab5a45-74d8-4d92-a43b-45dfcc6a9b93 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:02.571578 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1368/2755] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:25:02 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:25:02.577118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524f568-50bc-44e5-86af-cf175e918faa admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:02.596071 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524f568-50bc-44e5-86af-cf175e918faa admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:02.600940 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5861470-0aa5-443f-b5be-3f42a88864e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:02.601614 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5861470-0aa5-443f-b5be-3f42a88864e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:02.601915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5861470-0aa5-443f-b5be-3f42a88864e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:02.602192 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5861470-0aa5-443f-b5be-3f42a88864e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:02.602915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5861470-0aa5-443f-b5be-3f42a88864e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:02.604539 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5861470-0aa5-443f-b5be-3f42a88864e0 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:02.620703 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1388/2756] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:25:02.625405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab5a45-74d8-4d92-a43b-45dfcc6a9b93 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:02.635727 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab5a45-74d8-4d92-a43b-45dfcc6a9b93 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:02.641220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6b393-bead-4e81-9b99-738218f51c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:02.642183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6b393-bead-4e81-9b99-738218f51c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:02.642531 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6b393-bead-4e81-9b99-738218f51c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:02.642773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6b393-bead-4e81-9b99-738218f51c8c admin admin] PATH_INFO: `/v3/projects/fa44414678254f05a8e02d846b01de0b/users/30339941de9f42e691b669f6ff31e710/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:02.660146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6b393-bead-4e81-9b99-738218f51c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa44414678254f05a8e02d846b01de0b, user_id=30339941de9f42e691b669f6ff31e710, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:02.662884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6b393-bead-4e81-9b99-738218f51c8c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:02.680267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1369/2757] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:25:02 2022] PUT /identity/v3/projects/fa44414678254f05a8e02d846b01de0b/users/30339941de9f42e691b669f6ff31e710/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:02.685050 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5861470-0aa5-443f-b5be-3f42a88864e0 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:02.695833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5861470-0aa5-443f-b5be-3f42a88864e0 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:02.714886 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:02.782392 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4f7dd-4e25-4b95-974d-f6c80e5f598e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:02.784346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4f7dd-4e25-4b95-974d-f6c80e5f598e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:02.784969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4f7dd-4e25-4b95-974d-f6c80e5f598e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:02.785409 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4f7dd-4e25-4b95-974d-f6c80e5f598e admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:02.786987 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4f7dd-4e25-4b95-974d-f6c80e5f598e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:02.791036 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4f7dd-4e25-4b95-974d-f6c80e5f598e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:02.811764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1389/2758] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:25:02.815997 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6b393-bead-4e81-9b99-738218f51c8c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:02.833815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6b393-bead-4e81-9b99-738218f51c8c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:02.844370 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffa339-9dfa-4484-b2e8-78c46c164e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:02.844565 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffa339-9dfa-4484-b2e8-78c46c164e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:02.844836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffa339-9dfa-4484-b2e8-78c46c164e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:02.845114 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffa339-9dfa-4484-b2e8-78c46c164e8b admin admin] PATH_INFO: `/v3/projects/fa44414678254f05a8e02d846b01de0b/users/30339941de9f42e691b669f6ff31e710/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:02.847222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ffa339-9dfa-4484-b2e8-78c46c164e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa44414678254f05a8e02d846b01de0b, user_id=30339941de9f42e691b669f6ff31e710, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:02.849923 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ffa339-9dfa-4484-b2e8-78c46c164e8b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:02.862566 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1370/2759] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:25:02 2022] PUT /identity/v3/projects/fa44414678254f05a8e02d846b01de0b/users/30339941de9f42e691b669f6ff31e710/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:25:02.874715 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35dbf6-0ad8-47ad-a713-462f810f4b02 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:02.875026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35dbf6-0ad8-47ad-a713-462f810f4b02 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:02.876111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35dbf6-0ad8-47ad-a713-462f810f4b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:02.907875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4c35dbf6-0ad8-47ad-a713-462f810f4b02 None None] MFA Rules not processed for user `30339941de9f42e691b669f6ff31e710`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:02.909864 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4c35dbf6-0ad8-47ad-a713-462f810f4b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:03.011203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1390/2760] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:03.025579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffa339-9dfa-4484-b2e8-78c46c164e8b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:03.027614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-31ffa339-9dfa-4484-b2e8-78c46c164e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:03.101502 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffa339-9dfa-4484-b2e8-78c46c164e8b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:03.105643 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf685e5b-7ce4-4188-bfca-082e78033560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:03.106906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf685e5b-7ce4-4188-bfca-082e78033560 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:03.107198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf685e5b-7ce4-4188-bfca-082e78033560 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:03.107477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf685e5b-7ce4-4188-bfca-082e78033560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:03.124956 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf685e5b-7ce4-4188-bfca-082e78033560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:03.126716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf685e5b-7ce4-4188-bfca-082e78033560 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:03.138699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1371/2761] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:25:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:25:03.214592 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35dbf6-0ad8-47ad-a713-462f810f4b02 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:03.225279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35dbf6-0ad8-47ad-a713-462f810f4b02 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:03.230501 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fe58a-146d-464d-a5f0-2c6d856722e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:03.231641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fe58a-146d-464d-a5f0-2c6d856722e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:03.231923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fe58a-146d-464d-a5f0-2c6d856722e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:03.232390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fe58a-146d-464d-a5f0-2c6d856722e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:03.316479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fe58a-146d-464d-a5f0-2c6d856722e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:03.318951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fe58a-146d-464d-a5f0-2c6d856722e4 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:03.336224 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1391/2762] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:25:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:25:03.664320 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf685e5b-7ce4-4188-bfca-082e78033560 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:03.727505 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:03.753358 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf685e5b-7ce4-4188-bfca-082e78033560 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:03.759061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:03.760375 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:03.760663 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:03.761096 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] PATH_INFO: `/v3/users/ae37c581488e4b23ba3196cfdf479e76` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:03.762717 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae37c581488e4b23ba3196cfdf479e76)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:03.766513 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:03.827095 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae37c581488e4b23ba3196cfdf479e76'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:03.835560 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae37c581488e4b23ba3196cfdf479e76'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:03.849049 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae37c581488e4b23ba3196cfdf479e76'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:03.849915 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae37c581488e4b23ba3196cfdf479e76'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:03.859490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae37c581488e4b23ba3196cfdf479e76'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:03.867063 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1372/2763] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:25:03 2022] DELETE /identity/v3/users/ae37c581488e4b23ba3196cfdf479e76 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:03.871756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97daa05-8e1a-4ad0-8e20-7fa1eb82433f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:03.872935 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97daa05-8e1a-4ad0-8e20-7fa1eb82433f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:03.873051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97daa05-8e1a-4ad0-8e20-7fa1eb82433f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:03.893814 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f97daa05-8e1a-4ad0-8e20-7fa1eb82433f None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:03.894885 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f97daa05-8e1a-4ad0-8e20-7fa1eb82433f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:03.953941 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:03.985622 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1392/2764] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:25:03.994519 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:03.996608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:04.091700 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d0934-2d62-4460-86dc-38a9a314cd4e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:04.095476 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9b2c0-df27-40ed-becc-43f823ff6ae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:04.096277 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9b2c0-df27-40ed-becc-43f823ff6ae2 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:04.096561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9b2c0-df27-40ed-becc-43f823ff6ae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:04.096836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9b2c0-df27-40ed-becc-43f823ff6ae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:04.110379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9b2c0-df27-40ed-becc-43f823ff6ae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:04.111341 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9b2c0-df27-40ed-becc-43f823ff6ae2 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:04.123776 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1373/2765] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:25:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:25:04.698923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97daa05-8e1a-4ad0-8e20-7fa1eb82433f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:04.701812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f97daa05-8e1a-4ad0-8e20-7fa1eb82433f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:04.724652 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97daa05-8e1a-4ad0-8e20-7fa1eb82433f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:04.729757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:04.730665 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:04.730937 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:04.731258 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] PATH_INFO: `/v3/projects/f1c27a26ad3f4d76927193bf22012dd3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:04.732631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1c27a26ad3f4d76927193bf22012dd3)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:04.734696 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:04.817996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1c27a26ad3f4d76927193bf22012dd3', 'request_id': 'req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:04.826027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1c27a26ad3f4d76927193bf22012dd3', 'request_id': 'req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:04.833120 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1c27a26ad3f4d76927193bf22012dd3', 'request_id': 'req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:04.849912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] The token cache is being invalidate because project f1c27a26ad3f4d76927193bf22012dd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:25:04.850278 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:04.851127 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:04.852876 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1393/2766] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:25:04 2022] DELETE /identity/v3/projects/f1c27a26ad3f4d76927193bf22012dd3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:25:04.857649 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9b2c0-df27-40ed-becc-43f823ff6ae2 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:04.877633 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9b2c0-df27-40ed-becc-43f823ff6ae2 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:04.950850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9051c05-84e6-431b-9c16-c84179252000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:04.951742 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9051c05-84e6-431b-9c16-c84179252000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:04.952017 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9051c05-84e6-431b-9c16-c84179252000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:04.952287 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9051c05-84e6-431b-9c16-c84179252000 admin admin] PATH_INFO: `/v3/users/30339941de9f42e691b669f6ff31e710` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:04.953558 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9051c05-84e6-431b-9c16-c84179252000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30339941de9f42e691b669f6ff31e710)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:04.955582 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9051c05-84e6-431b-9c16-c84179252000 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:04.971679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:05.018065 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9051c05-84e6-431b-9c16-c84179252000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30339941de9f42e691b669f6ff31e710'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:05.029001 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9051c05-84e6-431b-9c16-c84179252000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30339941de9f42e691b669f6ff31e710'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:05.048660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9051c05-84e6-431b-9c16-c84179252000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30339941de9f42e691b669f6ff31e710'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:05.049438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9051c05-84e6-431b-9c16-c84179252000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30339941de9f42e691b669f6ff31e710'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:05.062564 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9051c05-84e6-431b-9c16-c84179252000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30339941de9f42e691b669f6ff31e710'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:05.071045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1374/2767] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:25:04 2022] DELETE /identity/v3/users/30339941de9f42e691b669f6ff31e710 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:05.484810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:05.486530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:05.501329 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4ff07-d69c-495a-8615-be9dac6f8d9e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:05.578618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865a926-d04e-4e65-b559-b1d2020f8f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:05.579488 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865a926-d04e-4e65-b559-b1d2020f8f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:05.579764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865a926-d04e-4e65-b559-b1d2020f8f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:05.580045 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865a926-d04e-4e65-b559-b1d2020f8f59 admin admin] PATH_INFO: `/v3/projects/fa44414678254f05a8e02d846b01de0b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:05.581240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865a926-d04e-4e65-b559-b1d2020f8f59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa44414678254f05a8e02d846b01de0b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:05.583214 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865a926-d04e-4e65-b559-b1d2020f8f59 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:05.638689 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6865a926-d04e-4e65-b559-b1d2020f8f59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa44414678254f05a8e02d846b01de0b', 'request_id': 'req-6865a926-d04e-4e65-b559-b1d2020f8f59'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:05.646197 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6865a926-d04e-4e65-b559-b1d2020f8f59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa44414678254f05a8e02d846b01de0b', 'request_id': 'req-6865a926-d04e-4e65-b559-b1d2020f8f59'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:05.652396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6865a926-d04e-4e65-b559-b1d2020f8f59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa44414678254f05a8e02d846b01de0b', 'request_id': 'req-6865a926-d04e-4e65-b559-b1d2020f8f59'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:05.656715 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:05.670300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6865a926-d04e-4e65-b559-b1d2020f8f59 admin admin] The token cache is being invalidate because project fa44414678254f05a8e02d846b01de0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:25:05.670702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6865a926-d04e-4e65-b559-b1d2020f8f59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:05.671275 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6865a926-d04e-4e65-b559-b1d2020f8f59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:05.673229 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1394/2768] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:25:05 2022] DELETE /identity/v3/projects/fa44414678254f05a8e02d846b01de0b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:05.680207 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e95224-361b-4ad2-923b-7f925fd47a05 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:05.680525 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e95224-361b-4ad2-923b-7f925fd47a05 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:05.680801 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e95224-361b-4ad2-923b-7f925fd47a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:05.700039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-06e95224-361b-4ad2-923b-7f925fd47a05 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:05.701678 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-06e95224-361b-4ad2-923b-7f925fd47a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:05.766900 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1375/2769] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:05.774743 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351ab9e-4300-40ab-bcbf-f28c4682505b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:05.775058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351ab9e-4300-40ab-bcbf-f28c4682505b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:05.775307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351ab9e-4300-40ab-bcbf-f28c4682505b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:05.799173 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c351ab9e-4300-40ab-bcbf-f28c4682505b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:05.800388 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c351ab9e-4300-40ab-bcbf-f28c4682505b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:05.809067 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1395/2770] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:05.814716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e95224-361b-4ad2-923b-7f925fd47a05 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:05.825309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e95224-361b-4ad2-923b-7f925fd47a05 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:05.830223 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a707ddc-1152-470c-b506-ad9d3248e3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:05.830884 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a707ddc-1152-470c-b506-ad9d3248e3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:05.831190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a707ddc-1152-470c-b506-ad9d3248e3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:05.831467 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a707ddc-1152-470c-b506-ad9d3248e3ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:05.832190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a707ddc-1152-470c-b506-ad9d3248e3ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:25:05.832491 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a707ddc-1152-470c-b506-ad9d3248e3ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:05.834146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a707ddc-1152-470c-b506-ad9d3248e3ca admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:05.853946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1376/2771] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:25:05.859084 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351ab9e-4300-40ab-bcbf-f28c4682505b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:05.868762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351ab9e-4300-40ab-bcbf-f28c4682505b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:05.874065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf977558-ba72-47a5-b1eb-63f5c8f566ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:05.874729 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977558-ba72-47a5-b1eb-63f5c8f566ce admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:05.875014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977558-ba72-47a5-b1eb-63f5c8f566ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:05.875291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977558-ba72-47a5-b1eb-63f5c8f566ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:05.875984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf977558-ba72-47a5-b1eb-63f5c8f566ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:05.877977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf977558-ba72-47a5-b1eb-63f5c8f566ce admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:05.909419 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1396/2772] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:25:05 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:25:05.914615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a707ddc-1152-470c-b506-ad9d3248e3ca admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:05.928391 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a707ddc-1152-470c-b506-ad9d3248e3ca admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:06.009089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6bdcee-ef50-447c-8643-900b5fc5a977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:06.009799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6bdcee-ef50-447c-8643-900b5fc5a977 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:06.010087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6bdcee-ef50-447c-8643-900b5fc5a977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:06.010359 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6bdcee-ef50-447c-8643-900b5fc5a977 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:06.011067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6bdcee-ef50-447c-8643-900b5fc5a977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:06.013548 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6bdcee-ef50-447c-8643-900b5fc5a977 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:06.044202 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1377/2773] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:25:05 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:25:06.045212 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf977558-ba72-47a5-b1eb-63f5c8f566ce admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:06.064007 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf977558-ba72-47a5-b1eb-63f5c8f566ce admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:06.069145 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6be7a4-c523-4a8f-8cdb-8f98accd10db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:06.069900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6be7a4-c523-4a8f-8cdb-8f98accd10db admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:06.070166 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6be7a4-c523-4a8f-8cdb-8f98accd10db admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:06.070538 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6be7a4-c523-4a8f-8cdb-8f98accd10db admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:06.071338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6be7a4-c523-4a8f-8cdb-8f98accd10db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:06.072996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6be7a4-c523-4a8f-8cdb-8f98accd10db admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:06.093965 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1397/2774] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:25:06.098992 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6bdcee-ef50-447c-8643-900b5fc5a977 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:06.111934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6bdcee-ef50-447c-8643-900b5fc5a977 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:06.117575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167797a-1845-4161-960f-91e1e8c835c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:06.118297 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167797a-1845-4161-960f-91e1e8c835c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:06.118574 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167797a-1845-4161-960f-91e1e8c835c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:06.118867 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167797a-1845-4161-960f-91e1e8c835c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:06.119569 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167797a-1845-4161-960f-91e1e8c835c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:06.121192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167797a-1845-4161-960f-91e1e8c835c3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:06.125509 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:06.138263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1378/2775] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:25:06.142044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6be7a4-c523-4a8f-8cdb-8f98accd10db admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:06.152547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6be7a4-c523-4a8f-8cdb-8f98accd10db admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:06.158350 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e9c81-c5c8-42f1-b07f-0c2c58e038bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:06.159249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e9c81-c5c8-42f1-b07f-0c2c58e038bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:06.159535 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e9c81-c5c8-42f1-b07f-0c2c58e038bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:06.159813 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e9c81-c5c8-42f1-b07f-0c2c58e038bf admin admin] PATH_INFO: `/v3/projects/01705ff6c8c44c77a1ff7676007fa271/users/e139819c99e34f92b4bdf0329f873512/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:06.180631 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e9c81-c5c8-42f1-b07f-0c2c58e038bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=01705ff6c8c44c77a1ff7676007fa271, user_id=e139819c99e34f92b4bdf0329f873512, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:06.183408 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e9c81-c5c8-42f1-b07f-0c2c58e038bf admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:06.200694 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1398/2776] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:25:06 2022] PUT /identity/v3/projects/01705ff6c8c44c77a1ff7676007fa271/users/e139819c99e34f92b4bdf0329f873512/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:06.211322 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090034-ec1e-49de-8230-8b15de6845ba None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:06.211599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090034-ec1e-49de-8230-8b15de6845ba None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:06.211943 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090034-ec1e-49de-8230-8b15de6845ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:06.248893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-09090034-ec1e-49de-8230-8b15de6845ba None None] MFA Rules not processed for user `e139819c99e34f92b4bdf0329f873512`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:06.250175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-09090034-ec1e-49de-8230-8b15de6845ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:06.337055 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1379/2777] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:06.349757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e9c81-c5c8-42f1-b07f-0c2c58e038bf admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:06.351738 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-137e9c81-c5c8-42f1-b07f-0c2c58e038bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:06.433093 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e9c81-c5c8-42f1-b07f-0c2c58e038bf admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:06.437762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e712ef9-200c-43c8-99d4-f2d906a412c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:06.438434 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e712ef9-200c-43c8-99d4-f2d906a412c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:06.438717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e712ef9-200c-43c8-99d4-f2d906a412c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:06.438989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e712ef9-200c-43c8-99d4-f2d906a412c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:06.455527 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e712ef9-200c-43c8-99d4-f2d906a412c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:06.457011 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e712ef9-200c-43c8-99d4-f2d906a412c7 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:06.471307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1399/2778] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:25:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:25:14.117792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09090034-ec1e-49de-8230-8b15de6845ba None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:14.188993 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09090034-ec1e-49de-8230-8b15de6845ba None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:14.193680 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:14.194584 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:14.194889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:14.195183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] PATH_INFO: `/v3/users/e139819c99e34f92b4bdf0329f873512` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:14.196352 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e139819c99e34f92b4bdf0329f873512)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:14.198318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:14.248876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e139819c99e34f92b4bdf0329f873512'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:14.256353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e139819c99e34f92b4bdf0329f873512'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:14.269092 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e139819c99e34f92b4bdf0329f873512'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:14.269902 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e139819c99e34f92b4bdf0329f873512'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:14.278402 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e139819c99e34f92b4bdf0329f873512'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:14.286258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1380/2779] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:25:14 2022] DELETE /identity/v3/users/e139819c99e34f92b4bdf0329f873512 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:14.291040 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe1521-9bd3-4331-aa82-fc5f7f47ffcc None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:14.291541 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe1521-9bd3-4331-aa82-fc5f7f47ffcc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:14.291632 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe1521-9bd3-4331-aa82-fc5f7f47ffcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:14.308956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-cbfe1521-9bd3-4331-aa82-fc5f7f47ffcc None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:14.310122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cbfe1521-9bd3-4331-aa82-fc5f7f47ffcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:14.378066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1400/2780] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:14.388353 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:14.389739 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:14.399553 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:14.460698 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f56e9-99bf-431a-a92d-8826cf206fbc admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:14.465878 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96ca64-8bb7-4b14-a97e-191d790e1533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:14.466723 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96ca64-8bb7-4b14-a97e-191d790e1533 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:14.467004 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96ca64-8bb7-4b14-a97e-191d790e1533 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:14.467279 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96ca64-8bb7-4b14-a97e-191d790e1533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:14.476762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96ca64-8bb7-4b14-a97e-191d790e1533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:14.478162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96ca64-8bb7-4b14-a97e-191d790e1533 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:14.488843 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1381/2781] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:25:14.957026 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe1521-9bd3-4331-aa82-fc5f7f47ffcc None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:14.958469 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cbfe1521-9bd3-4331-aa82-fc5f7f47ffcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:14.972474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe1521-9bd3-4331-aa82-fc5f7f47ffcc None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:14.983443 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9366b76-e62a-42ba-b63b-53167bd3ffb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:14.984304 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9366b76-e62a-42ba-b63b-53167bd3ffb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:14.984540 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9366b76-e62a-42ba-b63b-53167bd3ffb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:14.984816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9366b76-e62a-42ba-b63b-53167bd3ffb7 admin admin] PATH_INFO: `/v3/projects/01705ff6c8c44c77a1ff7676007fa271` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:14.986551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9366b76-e62a-42ba-b63b-53167bd3ffb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01705ff6c8c44c77a1ff7676007fa271)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:14.988775 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9366b76-e62a-42ba-b63b-53167bd3ffb7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:15.024646 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:15.039448 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c9366b76-e62a-42ba-b63b-53167bd3ffb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01705ff6c8c44c77a1ff7676007fa271', 'request_id': 'req-c9366b76-e62a-42ba-b63b-53167bd3ffb7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:15.046298 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c9366b76-e62a-42ba-b63b-53167bd3ffb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01705ff6c8c44c77a1ff7676007fa271', 'request_id': 'req-c9366b76-e62a-42ba-b63b-53167bd3ffb7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:15.051844 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c9366b76-e62a-42ba-b63b-53167bd3ffb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01705ff6c8c44c77a1ff7676007fa271', 'request_id': 'req-c9366b76-e62a-42ba-b63b-53167bd3ffb7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:15.065320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c9366b76-e62a-42ba-b63b-53167bd3ffb7 admin admin] The token cache is being invalidate because project 01705ff6c8c44c77a1ff7676007fa271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:25:15.065637 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c9366b76-e62a-42ba-b63b-53167bd3ffb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:15.066396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c9366b76-e62a-42ba-b63b-53167bd3ffb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:15.068052 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1401/2782] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:25:14 2022] DELETE /identity/v3/projects/01705ff6c8c44c77a1ff7676007fa271 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:15.073380 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42b41b-bcf2-4705-9a7f-51816a6c8921 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.073664 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42b41b-bcf2-4705-9a7f-51816a6c8921 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.073974 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42b41b-bcf2-4705-9a7f-51816a6c8921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.092716 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ff42b41b-bcf2-4705-9a7f-51816a6c8921 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:15.094263 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ff42b41b-bcf2-4705-9a7f-51816a6c8921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:15.158432 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1382/2783] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:15.167481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a600a-2d16-42d3-877e-74e18ecccb07 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.167771 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a600a-2d16-42d3-877e-74e18ecccb07 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.168051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a600a-2d16-42d3-877e-74e18ecccb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.190723 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-072a600a-2d16-42d3-877e-74e18ecccb07 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:15.191929 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-072a600a-2d16-42d3-877e-74e18ecccb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:15.201836 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1402/2784] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:15.205424 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42b41b-bcf2-4705-9a7f-51816a6c8921 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:15.216752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42b41b-bcf2-4705-9a7f-51816a6c8921 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:15.221604 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ffef2-15e5-4de0-ae27-a97842cefc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:15.222304 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ffef2-15e5-4de0-ae27-a97842cefc90 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.222587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ffef2-15e5-4de0-ae27-a97842cefc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.222861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ffef2-15e5-4de0-ae27-a97842cefc90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.223588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ffef2-15e5-4de0-ae27-a97842cefc90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:25:15.223932 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ffef2-15e5-4de0-ae27-a97842cefc90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:15.225534 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ffef2-15e5-4de0-ae27-a97842cefc90 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:15.238607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1383/2785] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:25:15.243759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a600a-2d16-42d3-877e-74e18ecccb07 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:15.253183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a600a-2d16-42d3-877e-74e18ecccb07 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:15.258751 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f5322-7f98-49c0-9dc7-fc7592375ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:15.259540 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f5322-7f98-49c0-9dc7-fc7592375ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.259898 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f5322-7f98-49c0-9dc7-fc7592375ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.260247 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f5322-7f98-49c0-9dc7-fc7592375ec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.261038 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f5322-7f98-49c0-9dc7-fc7592375ec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:15.264021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f5322-7f98-49c0-9dc7-fc7592375ec2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:15.294233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1403/2786] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:25:15 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:25:15.297806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ffef2-15e5-4de0-ae27-a97842cefc90 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:15.308677 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ffef2-15e5-4de0-ae27-a97842cefc90 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:15.369493 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabdede-7856-4d63-9c4a-38be47ae21fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:15.370205 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabdede-7856-4d63-9c4a-38be47ae21fe admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.370489 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabdede-7856-4d63-9c4a-38be47ae21fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.370762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabdede-7856-4d63-9c4a-38be47ae21fe admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.371474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabdede-7856-4d63-9c4a-38be47ae21fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:15.373453 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabdede-7856-4d63-9c4a-38be47ae21fe admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:15.398782 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1384/2787] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:25:15 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:25:15.400963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f5322-7f98-49c0-9dc7-fc7592375ec2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:15.413610 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f5322-7f98-49c0-9dc7-fc7592375ec2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:15.418334 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7389c3-80d0-447c-a501-acd0074f95fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:15.419549 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7389c3-80d0-447c-a501-acd0074f95fe admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.419943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7389c3-80d0-447c-a501-acd0074f95fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.420219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7389c3-80d0-447c-a501-acd0074f95fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.420956 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7389c3-80d0-447c-a501-acd0074f95fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:15.422618 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7389c3-80d0-447c-a501-acd0074f95fe admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:15.435075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1404/2788] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:25:15.439084 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabdede-7856-4d63-9c4a-38be47ae21fe admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:15.449348 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabdede-7856-4d63-9c4a-38be47ae21fe admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:15.454362 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb1fb1-3320-4c90-bf77-4e506e0d4029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:15.454799 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb1fb1-3320-4c90-bf77-4e506e0d4029 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.455056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb1fb1-3320-4c90-bf77-4e506e0d4029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.455327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb1fb1-3320-4c90-bf77-4e506e0d4029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.456044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb1fb1-3320-4c90-bf77-4e506e0d4029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:15.457715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb1fb1-3320-4c90-bf77-4e506e0d4029 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:15.470220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1385/2789] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:25:15.474544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7389c3-80d0-447c-a501-acd0074f95fe admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:15.486167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7389c3-80d0-447c-a501-acd0074f95fe admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:15.492843 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473cc55-e411-4e99-b5fc-55715f4c6b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:15.495756 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473cc55-e411-4e99-b5fc-55715f4c6b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.496807 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473cc55-e411-4e99-b5fc-55715f4c6b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.497043 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473cc55-e411-4e99-b5fc-55715f4c6b84 admin admin] PATH_INFO: `/v3/projects/1cdef5e7dcae4f039df5c82bf8cb257a/users/74fe3d38c6174992a23a696ee7ad8ceb/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.516112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473cc55-e411-4e99-b5fc-55715f4c6b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cdef5e7dcae4f039df5c82bf8cb257a, user_id=74fe3d38c6174992a23a696ee7ad8ceb, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:15.518830 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473cc55-e411-4e99-b5fc-55715f4c6b84 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:15.534480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1405/2790] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:25:15 2022] PUT /identity/v3/projects/1cdef5e7dcae4f039df5c82bf8cb257a/users/74fe3d38c6174992a23a696ee7ad8ceb/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:15.545960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125f0bb-c8b1-43e1-9cd6-70554679b597 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.546247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125f0bb-c8b1-43e1-9cd6-70554679b597 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.546571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125f0bb-c8b1-43e1-9cd6-70554679b597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.575138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d125f0bb-c8b1-43e1-9cd6-70554679b597 None None] MFA Rules not processed for user `74fe3d38c6174992a23a696ee7ad8ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:15.576806 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d125f0bb-c8b1-43e1-9cd6-70554679b597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:15.580926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:15.656073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1386/2791] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:15.667436 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473cc55-e411-4e99-b5fc-55715f4c6b84 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:15.669193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0473cc55-e411-4e99-b5fc-55715f4c6b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:15.727206 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473cc55-e411-4e99-b5fc-55715f4c6b84 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:15.731060 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5be89-2594-4402-adf5-ae83e65351b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:15.731702 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5be89-2594-4402-adf5-ae83e65351b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.731980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5be89-2594-4402-adf5-ae83e65351b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.732250 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5be89-2594-4402-adf5-ae83e65351b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.740551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5be89-2594-4402-adf5-ae83e65351b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:15.742017 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5be89-2594-4402-adf5-ae83e65351b2 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:15.750812 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1406/2792] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:25:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:25:15.777313 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125f0bb-c8b1-43e1-9cd6-70554679b597 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:15.837513 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f23a-b00e-4985-9325-ab9533b31e47 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.837832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f23a-b00e-4985-9325-ab9533b31e47 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.838115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f23a-b00e-4985-9325-ab9533b31e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.851656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125f0bb-c8b1-43e1-9cd6-70554679b597 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:15.856242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:15.856600 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e0f2f23a-b00e-4985-9325-ab9533b31e47 None None] MFA Rules not processed for user `bdcf4731b9944e79818294f38b17c5bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:15.857100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.857381 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.857654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] PATH_INFO: `/v3/users/74fe3d38c6174992a23a696ee7ad8ceb` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.858208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e0f2f23a-b00e-4985-9325-ab9533b31e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:15.858893 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74fe3d38c6174992a23a696ee7ad8ceb)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:15.860814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:15.909284 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74fe3d38c6174992a23a696ee7ad8ceb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:15.916602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1407/2793] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:25:15.917695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74fe3d38c6174992a23a696ee7ad8ceb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:15.931127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74fe3d38c6174992a23a696ee7ad8ceb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:15.931821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74fe3d38c6174992a23a696ee7ad8ceb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:15.939401 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74fe3d38c6174992a23a696ee7ad8ceb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:15.945407 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1387/2794] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:25:15 2022] DELETE /identity/v3/users/74fe3d38c6174992a23a696ee7ad8ceb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:15.950639 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfa0f0-5f07-45d6-80aa-3af329ad2fd2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:15.951112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfa0f0-5f07-45d6-80aa-3af329ad2fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:15.951393 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfa0f0-5f07-45d6-80aa-3af329ad2fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:15.957432 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:15.971762 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-70cfa0f0-5f07-45d6-80aa-3af329ad2fd2 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:15.972985 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-70cfa0f0-5f07-45d6-80aa-3af329ad2fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:16.033796 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1408/2795] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:16.043660 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:16.045242 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:16.079647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:16.113212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cabca8-d7f4-41c9-84ed-317baa8bae60 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:16.116821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ab519-bfd0-4e79-a774-eb4548abafd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:16.117469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ab519-bfd0-4e79-a774-eb4548abafd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:16.118168 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ab519-bfd0-4e79-a774-eb4548abafd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:16.118168 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ab519-bfd0-4e79-a774-eb4548abafd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:16.128886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ab519-bfd0-4e79-a774-eb4548abafd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:16.130447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ab519-bfd0-4e79-a774-eb4548abafd8 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:16.142150 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1388/2796] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:25:16.423987 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfa0f0-5f07-45d6-80aa-3af329ad2fd2 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:16.425903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-70cfa0f0-5f07-45d6-80aa-3af329ad2fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:16.440790 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfa0f0-5f07-45d6-80aa-3af329ad2fd2 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:16.450283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c63398-9c03-4c27-baf9-ec1c6c087708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:16.451363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c63398-9c03-4c27-baf9-ec1c6c087708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:16.451759 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c63398-9c03-4c27-baf9-ec1c6c087708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:16.452284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c63398-9c03-4c27-baf9-ec1c6c087708 admin admin] PATH_INFO: `/v3/projects/1cdef5e7dcae4f039df5c82bf8cb257a` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:16.454297 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c63398-9c03-4c27-baf9-ec1c6c087708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cdef5e7dcae4f039df5c82bf8cb257a)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:16.463763 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c63398-9c03-4c27-baf9-ec1c6c087708 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:16.524074 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-25c63398-9c03-4c27-baf9-ec1c6c087708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cdef5e7dcae4f039df5c82bf8cb257a', 'request_id': 'req-25c63398-9c03-4c27-baf9-ec1c6c087708'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:16.530473 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-25c63398-9c03-4c27-baf9-ec1c6c087708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cdef5e7dcae4f039df5c82bf8cb257a', 'request_id': 'req-25c63398-9c03-4c27-baf9-ec1c6c087708'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:16.536900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-25c63398-9c03-4c27-baf9-ec1c6c087708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cdef5e7dcae4f039df5c82bf8cb257a', 'request_id': 'req-25c63398-9c03-4c27-baf9-ec1c6c087708'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:16.553432 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-25c63398-9c03-4c27-baf9-ec1c6c087708 admin admin] The token cache is being invalidate because project 1cdef5e7dcae4f039df5c82bf8cb257a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:25:16.553748 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-25c63398-9c03-4c27-baf9-ec1c6c087708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:16.554407 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-25c63398-9c03-4c27-baf9-ec1c6c087708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:16.555960 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1409/2797] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:25:16 2022] DELETE /identity/v3/projects/1cdef5e7dcae4f039df5c82bf8cb257a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:16.562849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843cf91-e502-4bb0-ac8e-deaa1a5458a3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:16.563408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843cf91-e502-4bb0-ac8e-deaa1a5458a3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:16.563713 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843cf91-e502-4bb0-ac8e-deaa1a5458a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:16.585359 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7843cf91-e502-4bb0-ac8e-deaa1a5458a3 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:16.586720 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7843cf91-e502-4bb0-ac8e-deaa1a5458a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:16.650525 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1389/2798] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:16.658846 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6879d7-c326-453a-932f-cb3831013bf4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:16.659141 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6879d7-c326-453a-932f-cb3831013bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:16.659421 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6879d7-c326-453a-932f-cb3831013bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:16.680744 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-cc6879d7-c326-453a-932f-cb3831013bf4 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:16.682076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cc6879d7-c326-453a-932f-cb3831013bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:16.693617 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1410/2799] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:16.694987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843cf91-e502-4bb0-ac8e-deaa1a5458a3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:16.705172 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843cf91-e502-4bb0-ac8e-deaa1a5458a3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:16.709813 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52e1ca-4060-4e43-b0e7-68635260ff44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:16.710497 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52e1ca-4060-4e43-b0e7-68635260ff44 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:16.710777 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52e1ca-4060-4e43-b0e7-68635260ff44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:16.711054 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52e1ca-4060-4e43-b0e7-68635260ff44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:16.711774 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52e1ca-4060-4e43-b0e7-68635260ff44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:25:16.712087 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52e1ca-4060-4e43-b0e7-68635260ff44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:16.713697 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52e1ca-4060-4e43-b0e7-68635260ff44 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:16.727045 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1390/2800] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:25:16.732507 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6879d7-c326-453a-932f-cb3831013bf4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:16.743491 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6879d7-c326-453a-932f-cb3831013bf4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:16.748855 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4402900-76bd-4b8c-9ede-ff27e59116ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:16.749522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4402900-76bd-4b8c-9ede-ff27e59116ba admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:16.749845 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4402900-76bd-4b8c-9ede-ff27e59116ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:16.750121 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4402900-76bd-4b8c-9ede-ff27e59116ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:16.750824 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4402900-76bd-4b8c-9ede-ff27e59116ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:16.752736 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4402900-76bd-4b8c-9ede-ff27e59116ba admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:16.784092 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1411/2801] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:25:16 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:25:16.785126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52e1ca-4060-4e43-b0e7-68635260ff44 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:16.795922 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52e1ca-4060-4e43-b0e7-68635260ff44 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:16.856895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61aed8e-a731-4bbe-8d29-28fad44c36f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:16.857699 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61aed8e-a731-4bbe-8d29-28fad44c36f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:16.858034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61aed8e-a731-4bbe-8d29-28fad44c36f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:16.858305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61aed8e-a731-4bbe-8d29-28fad44c36f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:16.858995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61aed8e-a731-4bbe-8d29-28fad44c36f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:16.860888 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61aed8e-a731-4bbe-8d29-28fad44c36f3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:16.883236 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1391/2802] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:25:16 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:25:16.888075 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4402900-76bd-4b8c-9ede-ff27e59116ba admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:16.898410 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4402900-76bd-4b8c-9ede-ff27e59116ba admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:16.904849 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0d6a5-2fce-496b-9cdf-66eff32ce3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:16.905296 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0d6a5-2fce-496b-9cdf-66eff32ce3c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:16.905576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0d6a5-2fce-496b-9cdf-66eff32ce3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:16.905878 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0d6a5-2fce-496b-9cdf-66eff32ce3c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:16.906604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0d6a5-2fce-496b-9cdf-66eff32ce3c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:16.908215 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0d6a5-2fce-496b-9cdf-66eff32ce3c5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:16.920425 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1412/2803] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:25:16.925297 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61aed8e-a731-4bbe-8d29-28fad44c36f3 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:16.942414 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61aed8e-a731-4bbe-8d29-28fad44c36f3 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:16.950836 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838a3b2-b707-4153-8298-4a265fe0968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:16.953580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838a3b2-b707-4153-8298-4a265fe0968c admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:16.953580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838a3b2-b707-4153-8298-4a265fe0968c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:16.953580 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838a3b2-b707-4153-8298-4a265fe0968c admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:16.954766 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838a3b2-b707-4153-8298-4a265fe0968c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:16.957463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838a3b2-b707-4153-8298-4a265fe0968c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:16.969357 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1392/2804] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:25:16.973798 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0d6a5-2fce-496b-9cdf-66eff32ce3c5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:16.984410 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0d6a5-2fce-496b-9cdf-66eff32ce3c5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:16.989662 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d128138-8c93-4edb-84b9-39abeb865ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:16.990909 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d128138-8c93-4edb-84b9-39abeb865ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:16.991302 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d128138-8c93-4edb-84b9-39abeb865ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:16.991647 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d128138-8c93-4edb-84b9-39abeb865ad3 admin admin] PATH_INFO: `/v3/projects/90546acc85c14828b2f1e0a906037a9c/users/42f98b0f5afd497eb16ddd990dfcac26/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:17.008401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d128138-8c93-4edb-84b9-39abeb865ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90546acc85c14828b2f1e0a906037a9c, user_id=42f98b0f5afd497eb16ddd990dfcac26, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:17.011494 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d128138-8c93-4edb-84b9-39abeb865ad3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:17.035236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1413/2805] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:25:16 2022] PUT /identity/v3/projects/90546acc85c14828b2f1e0a906037a9c/users/42f98b0f5afd497eb16ddd990dfcac26/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:17.040390 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bb46e-8eed-4b5d-b71d-f0b083cc3083 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:17.040753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bb46e-8eed-4b5d-b71d-f0b083cc3083 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:17.041107 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bb46e-8eed-4b5d-b71d-f0b083cc3083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:17.062273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ac4bb46e-8eed-4b5d-b71d-f0b083cc3083 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:17.063934 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ac4bb46e-8eed-4b5d-b71d-f0b083cc3083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:17.099275 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:17.134302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1393/2806] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:17.144480 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d128138-8c93-4edb-84b9-39abeb865ad3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:17.146338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8d128138-8c93-4edb-84b9-39abeb865ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:17.175116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:17.210899 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d128138-8c93-4edb-84b9-39abeb865ad3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:17.214674 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb02097-be5d-47f8-b573-e513635db817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:17.215883 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb02097-be5d-47f8-b573-e513635db817 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:17.216159 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb02097-be5d-47f8-b573-e513635db817 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:17.216430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb02097-be5d-47f8-b573-e513635db817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:17.226557 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb02097-be5d-47f8-b573-e513635db817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:17.228048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb02097-be5d-47f8-b573-e513635db817 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:17.238465 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1414/2807] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:25:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:25:18.035123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26171566-f81f-42ae-b0d1-19db9c6dd50f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:18.035773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26171566-f81f-42ae-b0d1-19db9c6dd50f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:18.035773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26171566-f81f-42ae-b0d1-19db9c6dd50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:18.072370 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-26171566-f81f-42ae-b0d1-19db9c6dd50f None None] MFA Rules not processed for user `42f98b0f5afd497eb16ddd990dfcac26`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:18.074405 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-26171566-f81f-42ae-b0d1-19db9c6dd50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:18.157141 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1394/2808] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:18.168732 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb02097-be5d-47f8-b573-e513635db817 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:18.170274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-dfb02097-be5d-47f8-b573-e513635db817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:18.238691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb02097-be5d-47f8-b573-e513635db817 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:18.243568 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11ce34-e6e6-4afd-aaf3-edefd3745481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:18.244331 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11ce34-e6e6-4afd-aaf3-edefd3745481 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:18.244714 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11ce34-e6e6-4afd-aaf3-edefd3745481 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:18.245096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11ce34-e6e6-4afd-aaf3-edefd3745481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:18.255912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11ce34-e6e6-4afd-aaf3-edefd3745481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:18.257385 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11ce34-e6e6-4afd-aaf3-edefd3745481 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:18.267566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1415/2809] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:25:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:25:19.635986 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebb80e-6673-402c-a720-6592326f9334 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:19.636521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebb80e-6673-402c-a720-6592326f9334 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:19.636651 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebb80e-6673-402c-a720-6592326f9334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:19.655585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-65ebb80e-6673-402c-a720-6592326f9334 None None] MFA Rules not processed for user `42f98b0f5afd497eb16ddd990dfcac26`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:19.656953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-65ebb80e-6673-402c-a720-6592326f9334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:19.663953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1395/2810] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:28.262834 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=42 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:29.648977 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=47 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:35.454010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351836b-c2f9-495f-938e-4ef2af427b8e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:35.454590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351836b-c2f9-495f-938e-4ef2af427b8e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:35.454590 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351836b-c2f9-495f-938e-4ef2af427b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:35.473364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8351836b-c2f9-495f-938e-4ef2af427b8e None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:35.476094 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8351836b-c2f9-495f-938e-4ef2af427b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:35.538157 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1416/2811] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 18:25:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:35.585121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebb80e-6673-402c-a720-6592326f9334 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:35.586696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-65ebb80e-6673-402c-a720-6592326f9334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:35.662442 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebb80e-6673-402c-a720-6592326f9334 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:35.667399 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29333a8d-4649-4775-98b2-1952fe56af32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:35.668625 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29333a8d-4649-4775-98b2-1952fe56af32 service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:35.668911 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29333a8d-4649-4775-98b2-1952fe56af32 service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:35.669186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29333a8d-4649-4775-98b2-1952fe56af32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:35.677707 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29333a8d-4649-4775-98b2-1952fe56af32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:35.679591 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29333a8d-4649-4775-98b2-1952fe56af32 service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:35.688138 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1396/2812] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:25:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:25:45.531943 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=25 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:45.684980 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=32 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:47.442900 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351836b-c2f9-495f-938e-4ef2af427b8e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:47.444536 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8351836b-c2f9-495f-938e-4ef2af427b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:47.531758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351836b-c2f9-495f-938e-4ef2af427b8e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:47.537344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0001212-a3c5-4d76-bcca-1844ee0d2320 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:47.538114 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0001212-a3c5-4d76-bcca-1844ee0d2320 service placement] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:47.538458 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0001212-a3c5-4d76-bcca-1844ee0d2320 service placement] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:47.538799 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0001212-a3c5-4d76-bcca-1844ee0d2320 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:47.540969 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d0001212-a3c5-4d76-bcca-1844ee0d2320 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:47.555781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0001212-a3c5-4d76-bcca-1844ee0d2320 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:47.557816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0001212-a3c5-4d76-bcca-1844ee0d2320 service placement] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:47.571229 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1417/2813] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:25:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:25:53.910153 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda561d6-25a5-4c99-93eb-9fa87da41d89 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:53.910632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda561d6-25a5-4c99-93eb-9fa87da41d89 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:53.910726 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda561d6-25a5-4c99-93eb-9fa87da41d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:53.929002 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-dda561d6-25a5-4c99-93eb-9fa87da41d89 None None] MFA Rules not processed for user `90dcc2061d6b4987a1722fab33875b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:53.930194 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dda561d6-25a5-4c99-93eb-9fa87da41d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:53.937496 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1397/2814] 10.222.0.46 () {58 vars in 967 bytes} [Thu May 26 18:25:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:53.950192 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0001212-a3c5-4d76-bcca-1844ee0d2320 service placement] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:53.961080 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0001212-a3c5-4d76-bcca-1844ee0d2320 service placement] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:53.966741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0bce4-c3de-420b-a177-adc235d598e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd656b42c2acb498caa92c77a22fd7efe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:53.967376 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0bce4-c3de-420b-a177-adc235d598e1 service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:53.967660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0bce4-c3de-420b-a177-adc235d598e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:53.967936 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0bce4-c3de-420b-a177-adc235d598e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:53.975989 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0bce4-c3de-420b-a177-adc235d598e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:53.977588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0bce4-c3de-420b-a177-adc235d598e1 service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:53.985953 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1418/2815] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 18:25:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:25:55.916856 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda561d6-25a5-4c99-93eb-9fa87da41d89 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:55.918833 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dda561d6-25a5-4c99-93eb-9fa87da41d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:55.934913 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda561d6-25a5-4c99-93eb-9fa87da41d89 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:55.939673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b919834-26dc-45cf-912e-f6c1948bb29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:55.940543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b919834-26dc-45cf-912e-f6c1948bb29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:55.940830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b919834-26dc-45cf-912e-f6c1948bb29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:55.941105 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b919834-26dc-45cf-912e-f6c1948bb29f admin admin] PATH_INFO: `/v3/users/bdcf4731b9944e79818294f38b17c5bb` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:55.942365 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b919834-26dc-45cf-912e-f6c1948bb29f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdcf4731b9944e79818294f38b17c5bb)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:55.944377 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b919834-26dc-45cf-912e-f6c1948bb29f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:55.988688 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2b919834-26dc-45cf-912e-f6c1948bb29f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdcf4731b9944e79818294f38b17c5bb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:55.996410 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2b919834-26dc-45cf-912e-f6c1948bb29f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdcf4731b9944e79818294f38b17c5bb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:56.008563 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2b919834-26dc-45cf-912e-f6c1948bb29f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdcf4731b9944e79818294f38b17c5bb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:56.009186 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2b919834-26dc-45cf-912e-f6c1948bb29f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdcf4731b9944e79818294f38b17c5bb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:56.016769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2b919834-26dc-45cf-912e-f6c1948bb29f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdcf4731b9944e79818294f38b17c5bb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:56.022637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1398/2816] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:25:55 2022] DELETE /identity/v3/users/bdcf4731b9944e79818294f38b17c5bb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:56.199058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0bce4-c3de-420b-a177-adc235d598e1 service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:56.209754 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0bce4-c3de-420b-a177-adc235d598e1 service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:56.271993 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:56.272768 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:56.273046 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:56.273318 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459 admin admin] PATH_INFO: `/v3/projects/77a61f18a22c4890b3f8ae6b327a1ca5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:56.274529 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77a61f18a22c4890b3f8ae6b327a1ca5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:56.276496 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:56.324112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77a61f18a22c4890b3f8ae6b327a1ca5', 'request_id': 'req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:56.330289 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77a61f18a22c4890b3f8ae6b327a1ca5', 'request_id': 'req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:56.336126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77a61f18a22c4890b3f8ae6b327a1ca5', 'request_id': 'req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:56.348240 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459 admin admin] The token cache is being invalidate because project 77a61f18a22c4890b3f8ae6b327a1ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:25:56.348550 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:56.349176 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e1bf5308-e2f8-46bc-b19b-56f9b0eac459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:25:56.350755 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1419/2817] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:25:56 2022] DELETE /identity/v3/projects/77a61f18a22c4890b3f8ae6b327a1ca5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:56.356325 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f0a2f-adb1-4852-9064-50c2be6bb3ba None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:56.356607 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f0a2f-adb1-4852-9064-50c2be6bb3ba None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:56.356895 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f0a2f-adb1-4852-9064-50c2be6bb3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:56.375455 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-376f0a2f-adb1-4852-9064-50c2be6bb3ba None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:56.376648 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-376f0a2f-adb1-4852-9064-50c2be6bb3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:56.436599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1399/2818] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:56.444366 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ffe34-a314-467a-8efc-b20218c3d403 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:56.444651 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ffe34-a314-467a-8efc-b20218c3d403 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:56.444923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ffe34-a314-467a-8efc-b20218c3d403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:56.457411 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:56.464589 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-369ffe34-a314-467a-8efc-b20218c3d403 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:56.466781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-369ffe34-a314-467a-8efc-b20218c3d403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:56.474917 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1420/2819] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:56.479319 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f0a2f-adb1-4852-9064-50c2be6bb3ba None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:56.490416 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f0a2f-adb1-4852-9064-50c2be6bb3ba None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:56.495115 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01c818-c1d8-49c8-a3e2-ca8385ad09a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:56.496176 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01c818-c1d8-49c8-a3e2-ca8385ad09a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:56.496513 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01c818-c1d8-49c8-a3e2-ca8385ad09a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:56.496779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01c818-c1d8-49c8-a3e2-ca8385ad09a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:56.497502 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01c818-c1d8-49c8-a3e2-ca8385ad09a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:25:56.497832 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01c818-c1d8-49c8-a3e2-ca8385ad09a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:56.499448 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01c818-c1d8-49c8-a3e2-ca8385ad09a4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:56.512389 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1400/2820] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:25:56.516737 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ffe34-a314-467a-8efc-b20218c3d403 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:56.526049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ffe34-a314-467a-8efc-b20218c3d403 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:56.531367 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9cca9-bbc7-49ee-b967-a0f16acc00fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:56.532009 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9cca9-bbc7-49ee-b967-a0f16acc00fe admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:56.532285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9cca9-bbc7-49ee-b967-a0f16acc00fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:56.532555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9cca9-bbc7-49ee-b967-a0f16acc00fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:56.533363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9cca9-bbc7-49ee-b967-a0f16acc00fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:56.535292 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9cca9-bbc7-49ee-b967-a0f16acc00fe admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:56.562479 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1421/2821] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:25:56 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:25:56.566541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01c818-c1d8-49c8-a3e2-ca8385ad09a4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:56.576790 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01c818-c1d8-49c8-a3e2-ca8385ad09a4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:56.624338 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:25:56.634422 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308c6f4-c2d1-4d74-a323-9d1748ddd665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:56.635419 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308c6f4-c2d1-4d74-a323-9d1748ddd665 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:56.636054 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308c6f4-c2d1-4d74-a323-9d1748ddd665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:56.636376 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308c6f4-c2d1-4d74-a323-9d1748ddd665 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:56.637615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308c6f4-c2d1-4d74-a323-9d1748ddd665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:56.639795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308c6f4-c2d1-4d74-a323-9d1748ddd665 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:56.660062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1401/2822] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:25:56 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:25:56.664132 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9cca9-bbc7-49ee-b967-a0f16acc00fe admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:56.675526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9cca9-bbc7-49ee-b967-a0f16acc00fe admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:56.680719 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0cc6d-6213-4118-bb4d-1aae2d38913a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:56.681401 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0cc6d-6213-4118-bb4d-1aae2d38913a admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:56.681670 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0cc6d-6213-4118-bb4d-1aae2d38913a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:56.681974 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0cc6d-6213-4118-bb4d-1aae2d38913a admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:56.682783 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0cc6d-6213-4118-bb4d-1aae2d38913a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:56.684767 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0cc6d-6213-4118-bb4d-1aae2d38913a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:56.698437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1422/2823] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:25:56.700031 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308c6f4-c2d1-4d74-a323-9d1748ddd665 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:56.710157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308c6f4-c2d1-4d74-a323-9d1748ddd665 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:56.715530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cf927-8765-4d70-8015-63a923b030a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:56.716187 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cf927-8765-4d70-8015-63a923b030a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:56.716469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cf927-8765-4d70-8015-63a923b030a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:56.716745 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cf927-8765-4d70-8015-63a923b030a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:56.717457 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69cf927-8765-4d70-8015-63a923b030a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:56.719078 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69cf927-8765-4d70-8015-63a923b030a7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:56.729111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1402/2824] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:25:56.734175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0cc6d-6213-4118-bb4d-1aae2d38913a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:56.743996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0cc6d-6213-4118-bb4d-1aae2d38913a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:56.749361 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d077f84-003b-40c2-87e4-7fc1fe0433d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:56.750276 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d077f84-003b-40c2-87e4-7fc1fe0433d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:56.750557 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d077f84-003b-40c2-87e4-7fc1fe0433d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:56.750830 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d077f84-003b-40c2-87e4-7fc1fe0433d4 admin admin] PATH_INFO: `/v3/projects/564690a570ea4ec199dc599d288756b0/users/73559b7886614dd2a92e9c6c53307331/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:56.765254 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d077f84-003b-40c2-87e4-7fc1fe0433d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=564690a570ea4ec199dc599d288756b0, user_id=73559b7886614dd2a92e9c6c53307331, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:56.768010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d077f84-003b-40c2-87e4-7fc1fe0433d4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:56.781776 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1423/2825] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:25:56 2022] PUT /identity/v3/projects/564690a570ea4ec199dc599d288756b0/users/73559b7886614dd2a92e9c6c53307331/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:25:56.786671 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515fb05-c564-41bf-9e76-8dbd8034005b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:56.786965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515fb05-c564-41bf-9e76-8dbd8034005b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:56.787246 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515fb05-c564-41bf-9e76-8dbd8034005b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:56.805921 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9515fb05-c564-41bf-9e76-8dbd8034005b None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:25:56.807157 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9515fb05-c564-41bf-9e76-8dbd8034005b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:56.867321 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1403/2826] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:25:56.877267 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d077f84-003b-40c2-87e4-7fc1fe0433d4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:25:56.879023 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1d077f84-003b-40c2-87e4-7fc1fe0433d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:25:56.935761 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d077f84-003b-40c2-87e4-7fc1fe0433d4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:25:56.939474 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174cbdc-a075-425d-af89-1e345f9b82c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:25:56.940113 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174cbdc-a075-425d-af89-1e345f9b82c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:25:56.940396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174cbdc-a075-425d-af89-1e345f9b82c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:25:56.940673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174cbdc-a075-425d-af89-1e345f9b82c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:25:56.950199 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9174cbdc-a075-425d-af89-1e345f9b82c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:25:56.951653 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9174cbdc-a075-425d-af89-1e345f9b82c9 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:25:56.961229 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1424/2827] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:25:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:26:00.939978 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73aa2b6-c3e6-43e7-ad23-26ca2fa7ca40 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:00.940524 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73aa2b6-c3e6-43e7-ad23-26ca2fa7ca40 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:00.940524 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73aa2b6-c3e6-43e7-ad23-26ca2fa7ca40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:00.975124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f73aa2b6-c3e6-43e7-ad23-26ca2fa7ca40 None None] MFA Rules not processed for user `73559b7886614dd2a92e9c6c53307331`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:26:00.977119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f73aa2b6-c3e6-43e7-ad23-26ca2fa7ca40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:26:01.050656 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1404/2828] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:26:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:26:01.055352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174cbdc-a075-425d-af89-1e345f9b82c9 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:26:01.065926 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174cbdc-a075-425d-af89-1e345f9b82c9 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:26:01.071139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f1212-704b-47d7-9f39-c46a03477ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:26:01.072001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f1212-704b-47d7-9f39-c46a03477ffb admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:01.072346 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f1212-704b-47d7-9f39-c46a03477ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:01.072740 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f1212-704b-47d7-9f39-c46a03477ffb admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:01.074115 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f1212-704b-47d7-9f39-c46a03477ffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:26:01.076284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f1212-704b-47d7-9f39-c46a03477ffb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:26:01.119389 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1425/2829] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:26:01 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:26:01.123387 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73aa2b6-c3e6-43e7-ad23-26ca2fa7ca40 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:26:01.134608 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73aa2b6-c3e6-43e7-ad23-26ca2fa7ca40 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:26:01.200034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70685a8f-470a-42ba-98db-1d88b6566240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:26:01.200719 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70685a8f-470a-42ba-98db-1d88b6566240 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:01.201276 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70685a8f-470a-42ba-98db-1d88b6566240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:01.201562 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70685a8f-470a-42ba-98db-1d88b6566240 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:01.202323 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70685a8f-470a-42ba-98db-1d88b6566240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:26:01.204305 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70685a8f-470a-42ba-98db-1d88b6566240 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:26:01.215796 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:26:01.229291 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1405/2830] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:26:01 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:26:01.233991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f1212-704b-47d7-9f39-c46a03477ffb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:26:01.245175 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f1212-704b-47d7-9f39-c46a03477ffb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:26:01.250603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e4d23-2b4b-4a64-8abe-3c662e102d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:26:01.251282 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e4d23-2b4b-4a64-8abe-3c662e102d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:01.251566 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e4d23-2b4b-4a64-8abe-3c662e102d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:01.251842 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e4d23-2b4b-4a64-8abe-3c662e102d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:01.252576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e4d23-2b4b-4a64-8abe-3c662e102d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:26:01.254306 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e4d23-2b4b-4a64-8abe-3c662e102d41 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:26:01.266117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1426/2831] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:26:01.270344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70685a8f-470a-42ba-98db-1d88b6566240 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:26:01.281968 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70685a8f-470a-42ba-98db-1d88b6566240 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:26:01.286696 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe025a2-39ee-4e18-bd71-391b529c1888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:26:01.287587 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe025a2-39ee-4e18-bd71-391b529c1888 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:01.287983 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe025a2-39ee-4e18-bd71-391b529c1888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:01.288474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe025a2-39ee-4e18-bd71-391b529c1888 admin admin] PATH_INFO: `/v3/projects/8623d691c6384d53bb1fd6c9ecc3763f/users/8ac84f3961484c6dba931ff12a24e48c/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:01.305808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe025a2-39ee-4e18-bd71-391b529c1888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8623d691c6384d53bb1fd6c9ecc3763f, user_id=8ac84f3961484c6dba931ff12a24e48c, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:26:01.308889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe025a2-39ee-4e18-bd71-391b529c1888 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:26:01.325654 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1406/2832] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:26:01 2022] PUT /identity/v3/projects/8623d691c6384d53bb1fd6c9ecc3763f/users/8ac84f3961484c6dba931ff12a24e48c/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:26:01.329603 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e4d23-2b4b-4a64-8abe-3c662e102d41 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:26:01.340923 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e4d23-2b4b-4a64-8abe-3c662e102d41 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:26:01.400421 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4603f-f666-4f18-b93c-efc64bbfae30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:26:01.401104 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4603f-f666-4f18-b93c-efc64bbfae30 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:01.401456 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4603f-f666-4f18-b93c-efc64bbfae30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:01.402255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4603f-f666-4f18-b93c-efc64bbfae30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:01.403445 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4603f-f666-4f18-b93c-efc64bbfae30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:26:01.405073 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4603f-f666-4f18-b93c-efc64bbfae30 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:26:01.415673 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1427/2833] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:26:01.420112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe025a2-39ee-4e18-bd71-391b529c1888 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:26:01.431364 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe025a2-39ee-4e18-bd71-391b529c1888 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:26:01.436469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0b802-20ff-4631-aa79-9563bbf12659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:26:01.437515 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b802-20ff-4631-aa79-9563bbf12659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:01.437859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b802-20ff-4631-aa79-9563bbf12659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:01.438116 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b802-20ff-4631-aa79-9563bbf12659 admin admin] PATH_INFO: `/v3/projects/8623d691c6384d53bb1fd6c9ecc3763f/users/8ac84f3961484c6dba931ff12a24e48c/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:01.440122 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0b802-20ff-4631-aa79-9563bbf12659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8623d691c6384d53bb1fd6c9ecc3763f, user_id=8ac84f3961484c6dba931ff12a24e48c, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:26:01.442815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0b802-20ff-4631-aa79-9563bbf12659 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:26:01.456204 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1407/2834] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:26:01 2022] PUT /identity/v3/projects/8623d691c6384d53bb1fd6c9ecc3763f/users/8ac84f3961484c6dba931ff12a24e48c/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:26:05.994028 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e3344-6897-4868-8df5-3108e2cc1aca None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:05.994552 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e3344-6897-4868-8df5-3108e2cc1aca None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:05.995903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e3344-6897-4868-8df5-3108e2cc1aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:06.014574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:26:06.026437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-383e3344-6897-4868-8df5-3108e2cc1aca None None] MFA Rules not processed for user `8ac84f3961484c6dba931ff12a24e48c`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:26:06.027588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-383e3344-6897-4868-8df5-3108e2cc1aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:26:06.107981 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1428/2835] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:26:05 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:26:06.123466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0b802-20ff-4631-aa79-9563bbf12659 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:26:06.125705 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dfd0b802-20ff-4631-aa79-9563bbf12659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:26:06.207534 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0b802-20ff-4631-aa79-9563bbf12659 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:26:06.214360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127d46d-72d8-4ba8-8107-1d11e6a051a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:26:06.215431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127d46d-72d8-4ba8-8107-1d11e6a051a7 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:06.215814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127d46d-72d8-4ba8-8107-1d11e6a051a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:06.216146 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127d46d-72d8-4ba8-8107-1d11e6a051a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:06.279288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127d46d-72d8-4ba8-8107-1d11e6a051a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:26:06.281050 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127d46d-72d8-4ba8-8107-1d11e6a051a7 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:26:06.291192 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1408/2836] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:26:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:26:07.404323 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e7450-f432-4711-85a0-cb3f7a54753a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:07.404837 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e7450-f432-4711-85a0-cb3f7a54753a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:07.404837 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e7450-f432-4711-85a0-cb3f7a54753a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:07.425821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e53e7450-f432-4711-85a0-cb3f7a54753a None None] MFA Rules not processed for user `73559b7886614dd2a92e9c6c53307331`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:26:07.427251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e53e7450-f432-4711-85a0-cb3f7a54753a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:26:07.435654 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1429/2837] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:26:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:26:16.286340 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:INSERT=4,keystone:SELECT=73 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:26:17.418769 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=33 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:26:22.881511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4cb5d-3296-4c50-9d02-4f36bb645379 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:22.882046 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4cb5d-3296-4c50-9d02-4f36bb645379 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:22.882190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4cb5d-3296-4c50-9d02-4f36bb645379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:22.899259 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-80c4cb5d-3296-4c50-9d02-4f36bb645379 None None] MFA Rules not processed for user `42f98b0f5afd497eb16ddd990dfcac26`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:26:22.901006 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-80c4cb5d-3296-4c50-9d02-4f36bb645379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:26:22.954679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1409/2838] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:26:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:26:32.948060 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=25 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:26:39.878375 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e7450-f432-4711-85a0-cb3f7a54753a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:26:39.934378 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e7450-f432-4711-85a0-cb3f7a54753a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:26:39.937949 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eced751-1744-455c-9d1d-d8f541282493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:26:39.938588 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eced751-1744-455c-9d1d-d8f541282493 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:39.938872 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eced751-1744-455c-9d1d-d8f541282493 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:39.939142 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eced751-1744-455c-9d1d-d8f541282493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:39.948430 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eced751-1744-455c-9d1d-d8f541282493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:26:39.950179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eced751-1744-455c-9d1d-d8f541282493 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:26:39.959379 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1430/2839] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:26:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:26:41.568627 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4cb5d-3296-4c50-9d02-4f36bb645379 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:26:41.572669 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-80c4cb5d-3296-4c50-9d02-4f36bb645379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:26:41.593159 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4cb5d-3296-4c50-9d02-4f36bb645379 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:26:41.598947 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cebf98-080d-45b1-9935-78b14862a461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:26:41.599597 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cebf98-080d-45b1-9935-78b14862a461 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:26:41.599876 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cebf98-080d-45b1-9935-78b14862a461 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:26:41.600145 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cebf98-080d-45b1-9935-78b14862a461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:26:41.602190 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-56cebf98-080d-45b1-9935-78b14862a461 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:26:41.617036 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cebf98-080d-45b1-9935-78b14862a461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:26:41.618918 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cebf98-080d-45b1-9935-78b14862a461 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:26:41.633032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1410/2840] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:26:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:26:49.955279 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=26 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:26:51.627261 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:27:18.969434 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eced751-1744-455c-9d1d-d8f541282493 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:18.971271 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3eced751-1744-455c-9d1d-d8f541282493 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:19.039497 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eced751-1744-455c-9d1d-d8f541282493 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:19.044369 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a0cd7-afa8-4faa-9668-43ff084466fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:19.045245 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a0cd7-afa8-4faa-9668-43ff084466fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:19.045526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a0cd7-afa8-4faa-9668-43ff084466fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:19.045835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a0cd7-afa8-4faa-9668-43ff084466fa admin admin] PATH_INFO: `/v3/users/42f98b0f5afd497eb16ddd990dfcac26` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:19.047037 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a0cd7-afa8-4faa-9668-43ff084466fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=42f98b0f5afd497eb16ddd990dfcac26)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:19.048988 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a0cd7-afa8-4faa-9668-43ff084466fa admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:19.099424 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d09a0cd7-afa8-4faa-9668-43ff084466fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42f98b0f5afd497eb16ddd990dfcac26'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:19.108359 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d09a0cd7-afa8-4faa-9668-43ff084466fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42f98b0f5afd497eb16ddd990dfcac26'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:19.109040 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d09a0cd7-afa8-4faa-9668-43ff084466fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42f98b0f5afd497eb16ddd990dfcac26'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:19.109660 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d09a0cd7-afa8-4faa-9668-43ff084466fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42f98b0f5afd497eb16ddd990dfcac26'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:19.122597 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d09a0cd7-afa8-4faa-9668-43ff084466fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42f98b0f5afd497eb16ddd990dfcac26'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:19.137217 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1431/2841] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:27:18 2022] DELETE /identity/v3/users/42f98b0f5afd497eb16ddd990dfcac26 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:27:19.314461 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cebf98-080d-45b1-9935-78b14862a461 service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:19.325853 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cebf98-080d-45b1-9935-78b14862a461 service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:19.386800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:19.387703 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:19.387987 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:19.388258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f admin admin] PATH_INFO: `/v3/projects/90546acc85c14828b2f1e0a906037a9c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:19.389485 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f admin admin] RBAC: Authorizing `identity:delete_project(project_id=90546acc85c14828b2f1e0a906037a9c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:19.391585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:19.443591 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90546acc85c14828b2f1e0a906037a9c', 'request_id': 'req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:19.451431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90546acc85c14828b2f1e0a906037a9c', 'request_id': 'req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:19.461130 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90546acc85c14828b2f1e0a906037a9c', 'request_id': 'req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:19.472272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f admin admin] The token cache is being invalidate because project 90546acc85c14828b2f1e0a906037a9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:27:19.472759 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:19.473260 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d5cbe6fc-57d5-46c9-a7c3-394e26a6656f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:19.474908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1411/2842] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:27:19 2022] DELETE /identity/v3/projects/90546acc85c14828b2f1e0a906037a9c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:27:19.480741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33459635-9c9f-4ab2-b8e8-d36e2ec21dcf None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:19.481029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33459635-9c9f-4ab2-b8e8-d36e2ec21dcf None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:19.481312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33459635-9c9f-4ab2-b8e8-d36e2ec21dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:19.501203 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-33459635-9c9f-4ab2-b8e8-d36e2ec21dcf None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:27:19.502446 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-33459635-9c9f-4ab2-b8e8-d36e2ec21dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:19.563964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1432/2843] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:27:19.571784 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72b573-0513-4c7e-beef-fccbf381bbbc None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:19.572067 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72b573-0513-4c7e-beef-fccbf381bbbc None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:19.572339 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72b573-0513-4c7e-beef-fccbf381bbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:19.591914 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ac72b573-0513-4c7e-beef-fccbf381bbbc None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:27:19.593133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ac72b573-0513-4c7e-beef-fccbf381bbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:19.601222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1412/2844] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:27:19.606531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33459635-9c9f-4ab2-b8e8-d36e2ec21dcf None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:19.616547 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33459635-9c9f-4ab2-b8e8-d36e2ec21dcf None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:19.621364 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46591bfc-eac7-47ff-a037-28388c4b21d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:19.622437 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46591bfc-eac7-47ff-a037-28388c4b21d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:19.622753 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46591bfc-eac7-47ff-a037-28388c4b21d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:19.623032 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46591bfc-eac7-47ff-a037-28388c4b21d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:19.623764 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46591bfc-eac7-47ff-a037-28388c4b21d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:27:19.624076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46591bfc-eac7-47ff-a037-28388c4b21d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:19.625699 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46591bfc-eac7-47ff-a037-28388c4b21d7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:19.638712 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1433/2845] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:27:19.644088 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72b573-0513-4c7e-beef-fccbf381bbbc None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:19.655308 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72b573-0513-4c7e-beef-fccbf381bbbc None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:19.659981 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e3cab-e638-4d91-a5ed-fa661ec90188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:19.660651 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e3cab-e638-4d91-a5ed-fa661ec90188 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:19.660915 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e3cab-e638-4d91-a5ed-fa661ec90188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:19.661183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e3cab-e638-4d91-a5ed-fa661ec90188 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:19.661903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e3cab-e638-4d91-a5ed-fa661ec90188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:19.663795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e3cab-e638-4d91-a5ed-fa661ec90188 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:19.690798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1413/2846] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:27:19 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:27:19.695310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46591bfc-eac7-47ff-a037-28388c4b21d7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:19.705977 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46591bfc-eac7-47ff-a037-28388c4b21d7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:19.709558 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:27:19.768728 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45058bfa-96fe-44ad-9025-f53626159c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:19.769395 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45058bfa-96fe-44ad-9025-f53626159c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:19.769681 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45058bfa-96fe-44ad-9025-f53626159c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:19.769993 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45058bfa-96fe-44ad-9025-f53626159c8d admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:19.770691 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45058bfa-96fe-44ad-9025-f53626159c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:19.772633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45058bfa-96fe-44ad-9025-f53626159c8d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:19.795495 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1434/2847] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:27:19 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:27:19.799629 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e3cab-e638-4d91-a5ed-fa661ec90188 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:19.810573 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e3cab-e638-4d91-a5ed-fa661ec90188 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:19.815366 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642728f-d799-4e47-b7c5-082af0373a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:19.816124 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642728f-d799-4e47-b7c5-082af0373a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:19.816412 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642728f-d799-4e47-b7c5-082af0373a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:19.816690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642728f-d799-4e47-b7c5-082af0373a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:19.817408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642728f-d799-4e47-b7c5-082af0373a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:19.819056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642728f-d799-4e47-b7c5-082af0373a2d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:19.829871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1414/2848] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:27:19.834183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45058bfa-96fe-44ad-9025-f53626159c8d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:19.845410 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45058bfa-96fe-44ad-9025-f53626159c8d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:19.850151 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f71214-4a5d-445d-9a4e-0d5fd0c7cf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:19.850802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f71214-4a5d-445d-9a4e-0d5fd0c7cf3e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:19.851083 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f71214-4a5d-445d-9a4e-0d5fd0c7cf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:19.851403 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f71214-4a5d-445d-9a4e-0d5fd0c7cf3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:19.852193 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f71214-4a5d-445d-9a4e-0d5fd0c7cf3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:19.853996 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f71214-4a5d-445d-9a4e-0d5fd0c7cf3e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:19.865358 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1435/2849] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:27:19.869541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642728f-d799-4e47-b7c5-082af0373a2d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:19.880019 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642728f-d799-4e47-b7c5-082af0373a2d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:19.884616 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ebe39-d01e-4761-ab40-3d56c3d63675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:19.885903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ebe39-d01e-4761-ab40-3d56c3d63675 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:19.886196 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ebe39-d01e-4761-ab40-3d56c3d63675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:19.886490 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ebe39-d01e-4761-ab40-3d56c3d63675 admin admin] PATH_INFO: `/v3/projects/89af06cb5cdc44df8a8fabbd64c59008/users/c847016da8384c17af26652f8d1a3529/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:19.900988 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ebe39-d01e-4761-ab40-3d56c3d63675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89af06cb5cdc44df8a8fabbd64c59008, user_id=c847016da8384c17af26652f8d1a3529, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:19.904273 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ebe39-d01e-4761-ab40-3d56c3d63675 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:19.918343 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1415/2850] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:27:19 2022] PUT /identity/v3/projects/89af06cb5cdc44df8a8fabbd64c59008/users/c847016da8384c17af26652f8d1a3529/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:27:19.981555 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbb4b5-b5cf-40ca-a3db-8fbe4786324d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:19.982179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbb4b5-b5cf-40ca-a3db-8fbe4786324d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:19.982179 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbb4b5-b5cf-40ca-a3db-8fbe4786324d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:20.009162 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-69bbb4b5-b5cf-40ca-a3db-8fbe4786324d None None] MFA Rules not processed for user `c847016da8384c17af26652f8d1a3529`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:27:20.010415 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-69bbb4b5-b5cf-40ca-a3db-8fbe4786324d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:20.080802 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1436/2851] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:27:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:27:20.093029 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ebe39-d01e-4761-ab40-3d56c3d63675 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:20.094795 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4a4ebe39-d01e-4761-ab40-3d56c3d63675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:20.127166 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:27:20.155482 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ebe39-d01e-4761-ab40-3d56c3d63675 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:20.159016 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74f11c-a2cf-46db-8ffe-99da7b42841f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:20.159666 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74f11c-a2cf-46db-8ffe-99da7b42841f service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:20.159945 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74f11c-a2cf-46db-8ffe-99da7b42841f service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:20.160221 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74f11c-a2cf-46db-8ffe-99da7b42841f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:20.169266 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74f11c-a2cf-46db-8ffe-99da7b42841f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:20.170735 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74f11c-a2cf-46db-8ffe-99da7b42841f service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:20.179880 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1416/2852] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:27:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:27:27.269266 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbb4b5-b5cf-40ca-a3db-8fbe4786324d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:27.333274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbb4b5-b5cf-40ca-a3db-8fbe4786324d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:27.338077 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:27.339031 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:27.339360 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:27.340188 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] PATH_INFO: `/v3/users/c847016da8384c17af26652f8d1a3529` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:27.341517 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c847016da8384c17af26652f8d1a3529)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:27.343531 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:27.360778 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:27:27.394158 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c847016da8384c17af26652f8d1a3529'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:27.401320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c847016da8384c17af26652f8d1a3529'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:27.401966 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c847016da8384c17af26652f8d1a3529'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:27.403116 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c847016da8384c17af26652f8d1a3529'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:27.415340 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c847016da8384c17af26652f8d1a3529'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:27.428619 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1437/2853] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:27:27 2022] DELETE /identity/v3/users/c847016da8384c17af26652f8d1a3529 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:27:27.433100 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29e3b4-982c-4cca-96e4-ef415e15e8a5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:27.433754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29e3b4-982c-4cca-96e4-ef415e15e8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:27.434292 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29e3b4-982c-4cca-96e4-ef415e15e8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:27.453368 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5e29e3b4-982c-4cca-96e4-ef415e15e8a5 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:27:27.454596 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5e29e3b4-982c-4cca-96e4-ef415e15e8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:27.530788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1417/2854] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:27:27.542671 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:27.545058 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:27.614079 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a064d43-89a7-4103-80e9-0d15215e6b4b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:27.617716 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cb957-82c9-4fec-8726-464c392ac354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:27.618390 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cb957-82c9-4fec-8726-464c392ac354 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:27.618675 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cb957-82c9-4fec-8726-464c392ac354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:27.618951 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cb957-82c9-4fec-8726-464c392ac354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:27.629305 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cb957-82c9-4fec-8726-464c392ac354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:27.632284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cb957-82c9-4fec-8726-464c392ac354 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:27.644586 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1438/2855] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:27:27.925056 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29e3b4-982c-4cca-96e4-ef415e15e8a5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:27.926868 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5e29e3b4-982c-4cca-96e4-ef415e15e8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:27.949695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29e3b4-982c-4cca-96e4-ef415e15e8a5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:27.958779 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5007ed9-0a88-4cf1-8844-2678c528bffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:27.960673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5007ed9-0a88-4cf1-8844-2678c528bffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:27.961183 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5007ed9-0a88-4cf1-8844-2678c528bffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:27.961690 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5007ed9-0a88-4cf1-8844-2678c528bffe admin admin] PATH_INFO: `/v3/projects/89af06cb5cdc44df8a8fabbd64c59008` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:27.963466 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5007ed9-0a88-4cf1-8844-2678c528bffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=89af06cb5cdc44df8a8fabbd64c59008)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:27.965859 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5007ed9-0a88-4cf1-8844-2678c528bffe admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:28.015752 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b5007ed9-0a88-4cf1-8844-2678c528bffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89af06cb5cdc44df8a8fabbd64c59008', 'request_id': 'req-b5007ed9-0a88-4cf1-8844-2678c528bffe'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:28.023350 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b5007ed9-0a88-4cf1-8844-2678c528bffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89af06cb5cdc44df8a8fabbd64c59008', 'request_id': 'req-b5007ed9-0a88-4cf1-8844-2678c528bffe'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:28.030789 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b5007ed9-0a88-4cf1-8844-2678c528bffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89af06cb5cdc44df8a8fabbd64c59008', 'request_id': 'req-b5007ed9-0a88-4cf1-8844-2678c528bffe'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:28.041360 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b5007ed9-0a88-4cf1-8844-2678c528bffe admin admin] The token cache is being invalidate because project 89af06cb5cdc44df8a8fabbd64c59008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:27:28.041583 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b5007ed9-0a88-4cf1-8844-2678c528bffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:28.042251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b5007ed9-0a88-4cf1-8844-2678c528bffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:27:28.043773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1418/2856] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:27:27 2022] DELETE /identity/v3/projects/89af06cb5cdc44df8a8fabbd64c59008 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:27:28.048871 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e48ea-1a03-49c9-bd5c-ebec7696238c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:28.049185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e48ea-1a03-49c9-bd5c-ebec7696238c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:28.049460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e48ea-1a03-49c9-bd5c-ebec7696238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:28.067777 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5c5e48ea-1a03-49c9-bd5c-ebec7696238c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:27:28.069001 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5c5e48ea-1a03-49c9-bd5c-ebec7696238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:28.134796 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1439/2857] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:27:28.143662 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272f5c5-61c6-4496-89fe-1dfa97b1164c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:28.143951 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272f5c5-61c6-4496-89fe-1dfa97b1164c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:28.144257 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272f5c5-61c6-4496-89fe-1dfa97b1164c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:28.162541 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e272f5c5-61c6-4496-89fe-1dfa97b1164c None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:27:28.163737 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e272f5c5-61c6-4496-89fe-1dfa97b1164c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:28.172501 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1419/2858] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:27:28.176635 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e48ea-1a03-49c9-bd5c-ebec7696238c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:28.187291 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e48ea-1a03-49c9-bd5c-ebec7696238c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:28.191857 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22d77-6754-421b-a4e2-ca8f7ea61efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:28.192500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22d77-6754-421b-a4e2-ca8f7ea61efa admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:28.192781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22d77-6754-421b-a4e2-ca8f7ea61efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:28.193051 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22d77-6754-421b-a4e2-ca8f7ea61efa admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:28.194690 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c22d77-6754-421b-a4e2-ca8f7ea61efa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:27:28.195012 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c22d77-6754-421b-a4e2-ca8f7ea61efa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:28.196677 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c22d77-6754-421b-a4e2-ca8f7ea61efa admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:28.209053 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1440/2859] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 18:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:27:28.214127 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272f5c5-61c6-4496-89fe-1dfa97b1164c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:28.224585 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272f5c5-61c6-4496-89fe-1dfa97b1164c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:28.229599 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b91348-96d5-4f2e-883d-203c5ad9b164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:28.230347 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b91348-96d5-4f2e-883d-203c5ad9b164 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:28.230647 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b91348-96d5-4f2e-883d-203c5ad9b164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:28.230904 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b91348-96d5-4f2e-883d-203c5ad9b164 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:28.231814 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b91348-96d5-4f2e-883d-203c5ad9b164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:28.233468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b91348-96d5-4f2e-883d-203c5ad9b164 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:28.259468 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1420/2860] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:27:28 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:27:28.264391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22d77-6754-421b-a4e2-ca8f7ea61efa admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:28.274460 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22d77-6754-421b-a4e2-ca8f7ea61efa admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:28.295576 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:27:28.338111 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ba320-2a0a-48ef-a7ed-5b2675faf720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:28.338786 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ba320-2a0a-48ef-a7ed-5b2675faf720 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:28.339065 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ba320-2a0a-48ef-a7ed-5b2675faf720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:28.339339 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ba320-2a0a-48ef-a7ed-5b2675faf720 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:28.340735 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ba320-2a0a-48ef-a7ed-5b2675faf720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:28.342782 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ba320-2a0a-48ef-a7ed-5b2675faf720 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:28.366126 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1441/2861] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:27:28 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:27:28.370344 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b91348-96d5-4f2e-883d-203c5ad9b164 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:28.387162 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b91348-96d5-4f2e-883d-203c5ad9b164 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:28.392543 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7904c-68cf-4c5a-90d7-8d685cafc70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:28.393318 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7904c-68cf-4c5a-90d7-8d685cafc70c admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:28.393711 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7904c-68cf-4c5a-90d7-8d685cafc70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:28.394135 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7904c-68cf-4c5a-90d7-8d685cafc70c admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:28.395044 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7904c-68cf-4c5a-90d7-8d685cafc70c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:28.396753 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7904c-68cf-4c5a-90d7-8d685cafc70c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:28.408941 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1421/2862] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:27:28.413835 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ba320-2a0a-48ef-a7ed-5b2675faf720 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:28.425088 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ba320-2a0a-48ef-a7ed-5b2675faf720 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:28.430118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17bbaf-ce58-4472-ba52-07d81a6838b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:28.430772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17bbaf-ce58-4472-ba52-07d81a6838b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:28.431049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17bbaf-ce58-4472-ba52-07d81a6838b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:28.431320 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17bbaf-ce58-4472-ba52-07d81a6838b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:28.432161 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17bbaf-ce58-4472-ba52-07d81a6838b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:28.433892 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17bbaf-ce58-4472-ba52-07d81a6838b0 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:28.445300 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1442/2863] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:27:28.451030 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7904c-68cf-4c5a-90d7-8d685cafc70c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:28.461547 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7904c-68cf-4c5a-90d7-8d685cafc70c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:28.466925 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8478a-d141-4e68-a8a0-136ac15fbeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:28.467834 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8478a-d141-4e68-a8a0-136ac15fbeea admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:28.468091 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8478a-d141-4e68-a8a0-136ac15fbeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:28.468372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8478a-d141-4e68-a8a0-136ac15fbeea admin admin] PATH_INFO: `/v3/projects/6774c3ce4b094e19ad6da95e3744d9ed/users/7cde3488be6c4c8790841b7529123326/roles/e1a5192e7abc4046bc2c0673176ea1b6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:28.479995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:27:28.484927 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8478a-d141-4e68-a8a0-136ac15fbeea admin admin] RBAC: Authorizing `identity:create_grant(project_id=6774c3ce4b094e19ad6da95e3744d9ed, user_id=7cde3488be6c4c8790841b7529123326, role_id=e1a5192e7abc4046bc2c0673176ea1b6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:28.488533 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8478a-d141-4e68-a8a0-136ac15fbeea admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:28.502160 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1422/2864] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:27:28 2022] PUT /identity/v3/projects/6774c3ce4b094e19ad6da95e3744d9ed/users/7cde3488be6c4c8790841b7529123326/roles/e1a5192e7abc4046bc2c0673176ea1b6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:27:28.507533 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8fa99-f367-4f0a-8f9c-1e7539cd60f2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:28.507823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8fa99-f367-4f0a-8f9c-1e7539cd60f2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:28.508101 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8fa99-f367-4f0a-8f9c-1e7539cd60f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:28.526178 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-55f8fa99-f367-4f0a-8f9c-1e7539cd60f2 None None] MFA Rules not processed for user `3b98d845867d491c91aeca1e12cf3ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:27:28.528237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-55f8fa99-f367-4f0a-8f9c-1e7539cd60f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:28.590055 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1443/2865] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:27:28.600586 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8478a-d141-4e68-a8a0-136ac15fbeea admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:28.602685 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-26c8478a-d141-4e68-a8a0-136ac15fbeea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:28.664086 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8478a-d141-4e68-a8a0-136ac15fbeea admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:28.668845 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc72be6-dad8-4ff4-8b47-5dd817a760db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:28.669511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc72be6-dad8-4ff4-8b47-5dd817a760db service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:28.669826 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc72be6-dad8-4ff4-8b47-5dd817a760db service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:28.670103 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc72be6-dad8-4ff4-8b47-5dd817a760db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:28.680406 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc72be6-dad8-4ff4-8b47-5dd817a760db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:28.681894 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc72be6-dad8-4ff4-8b47-5dd817a760db service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:28.692747 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1423/2866] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:27:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:27:32.816751 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6ac7b-722c-4902-b66d-5c351c78e616 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:32.818062 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6ac7b-722c-4902-b66d-5c351c78e616 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:32.818853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6ac7b-722c-4902-b66d-5c351c78e616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:32.879008 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d7a6ac7b-722c-4902-b66d-5c351c78e616 None None] MFA Rules not processed for user `7cde3488be6c4c8790841b7529123326`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:27:32.880236 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d7a6ac7b-722c-4902-b66d-5c351c78e616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:32.964604 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1444/2867] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:27:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:27:32.970123 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc72be6-dad8-4ff4-8b47-5dd817a760db service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:32.980177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc72be6-dad8-4ff4-8b47-5dd817a760db service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:32.985133 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b735cf-32bb-47fd-a17b-14c076ae2769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:32.986142 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b735cf-32bb-47fd-a17b-14c076ae2769 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:32.986610 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b735cf-32bb-47fd-a17b-14c076ae2769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:32.987119 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b735cf-32bb-47fd-a17b-14c076ae2769 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:32.988026 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b735cf-32bb-47fd-a17b-14c076ae2769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:32.990249 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b735cf-32bb-47fd-a17b-14c076ae2769 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:33.025051 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1424/2868] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 18:27:32 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:27:33.030021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6ac7b-722c-4902-b66d-5c351c78e616 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:33.042991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6ac7b-722c-4902-b66d-5c351c78e616 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:33.082863 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:27:33.117554 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6e5c0-6d6f-4df4-b9ae-dcc89fb56e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:33.118344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6e5c0-6d6f-4df4-b9ae-dcc89fb56e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:33.118629 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6e5c0-6d6f-4df4-b9ae-dcc89fb56e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:33.118907 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6e5c0-6d6f-4df4-b9ae-dcc89fb56e70 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:33.119623 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6e5c0-6d6f-4df4-b9ae-dcc89fb56e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:33.121658 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6e5c0-6d6f-4df4-b9ae-dcc89fb56e70 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:33.145118 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1445/2869] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 18:27:33 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:27:33.149248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b735cf-32bb-47fd-a17b-14c076ae2769 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:33.161372 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b735cf-32bb-47fd-a17b-14c076ae2769 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:33.166142 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f3c2a-eb2d-4037-a623-6c089a877a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:33.166813 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f3c2a-eb2d-4037-a623-6c089a877a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:33.167090 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f3c2a-eb2d-4037-a623-6c089a877a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:33.167361 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f3c2a-eb2d-4037-a623-6c089a877a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:33.168815 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f3c2a-eb2d-4037-a623-6c089a877a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:33.170495 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f3c2a-eb2d-4037-a623-6c089a877a8e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:33.183477 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1425/2870] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:27:33.187722 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6e5c0-6d6f-4df4-b9ae-dcc89fb56e70 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:33.198372 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6e5c0-6d6f-4df4-b9ae-dcc89fb56e70 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:33.204284 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c266cf-f1e5-47de-ac57-78a13da9c518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:33.205308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c266cf-f1e5-47de-ac57-78a13da9c518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:33.205663 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c266cf-f1e5-47de-ac57-78a13da9c518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:33.206265 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c266cf-f1e5-47de-ac57-78a13da9c518 admin admin] PATH_INFO: `/v3/projects/1ca302a96a414709a24f15fd03edc0d6/users/36a8414532e24fc9badf492770dd75af/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:33.229715 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c266cf-f1e5-47de-ac57-78a13da9c518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca302a96a414709a24f15fd03edc0d6, user_id=36a8414532e24fc9badf492770dd75af, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:33.235307 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c266cf-f1e5-47de-ac57-78a13da9c518 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:33.254048 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1446/2871] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:27:33 2022] PUT /identity/v3/projects/1ca302a96a414709a24f15fd03edc0d6/users/36a8414532e24fc9badf492770dd75af/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:27:33.259960 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f3c2a-eb2d-4037-a623-6c089a877a8e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:33.273919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f3c2a-eb2d-4037-a623-6c089a877a8e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:33.369789 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a8fda-f7db-465a-8d82-22e90e9a053a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:33.370486 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a8fda-f7db-465a-8d82-22e90e9a053a admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:33.370800 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a8fda-f7db-465a-8d82-22e90e9a053a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:33.371081 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a8fda-f7db-465a-8d82-22e90e9a053a admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:33.371821 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a8fda-f7db-465a-8d82-22e90e9a053a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:33.373451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a8fda-f7db-465a-8d82-22e90e9a053a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:33.385408 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1426/2872] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 18:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:27:33.392343 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c266cf-f1e5-47de-ac57-78a13da9c518 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:33.404593 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c266cf-f1e5-47de-ac57-78a13da9c518 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:33.409574 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca172a-e700-4daa-8fc0-b73ca9935fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:33.410565 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca172a-e700-4daa-8fc0-b73ca9935fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:33.410912 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca172a-e700-4daa-8fc0-b73ca9935fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:33.411251 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca172a-e700-4daa-8fc0-b73ca9935fcc admin admin] PATH_INFO: `/v3/projects/1ca302a96a414709a24f15fd03edc0d6/users/36a8414532e24fc9badf492770dd75af/roles/584beecb022b422f820e77f7623ada5e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:33.413458 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca172a-e700-4daa-8fc0-b73ca9935fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca302a96a414709a24f15fd03edc0d6, user_id=36a8414532e24fc9badf492770dd75af, role_id=584beecb022b422f820e77f7623ada5e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:33.416255 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca172a-e700-4daa-8fc0-b73ca9935fcc admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:33.429288 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1447/2873] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 18:27:33 2022] PUT /identity/v3/projects/1ca302a96a414709a24f15fd03edc0d6/users/36a8414532e24fc9badf492770dd75af/roles/584beecb022b422f820e77f7623ada5e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:27:38.482724 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cae2d1-e3c3-48b3-9303-262e0eedfb30 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:38.483315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cae2d1-e3c3-48b3-9303-262e0eedfb30 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:38.483315 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cae2d1-e3c3-48b3-9303-262e0eedfb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:38.515632 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-57cae2d1-e3c3-48b3-9303-262e0eedfb30 None None] MFA Rules not processed for user `36a8414532e24fc9badf492770dd75af`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:27:38.517073 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-57cae2d1-e3c3-48b3-9303-262e0eedfb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:38.570254 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:27:38.602620 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1427/2874] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:27:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:27:38.619396 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca172a-e700-4daa-8fc0-b73ca9935fcc admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:38.622317 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f7ca172a-e700-4daa-8fc0-b73ca9935fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:38.712964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca172a-e700-4daa-8fc0-b73ca9935fcc admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:38.719877 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943c87d-ed6b-49ca-8115-afa93f6652b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:38.720551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c87d-ed6b-49ca-8115-afa93f6652b2 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:38.720832 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c87d-ed6b-49ca-8115-afa93f6652b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:38.721109 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c87d-ed6b-49ca-8115-afa93f6652b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:38.786922 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943c87d-ed6b-49ca-8115-afa93f6652b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:38.788553 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943c87d-ed6b-49ca-8115-afa93f6652b2 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:38.798551 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1448/2875] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:27:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:27:48.599013 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=9 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:27:48.793894 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=85,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:27:55.270267 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401ce54-6dd0-4504-912a-31896ce81ddb None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:55.271230 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401ce54-6dd0-4504-912a-31896ce81ddb None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:55.271592 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401ce54-6dd0-4504-912a-31896ce81ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:55.293330 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4401ce54-6dd0-4504-912a-31896ce81ddb None None] MFA Rules not processed for user `7cde3488be6c4c8790841b7529123326`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:27:55.294575 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4401ce54-6dd0-4504-912a-31896ce81ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:55.304928 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1428/2876] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:27:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:27:55.348562 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943c87d-ed6b-49ca-8115-afa93f6652b2 service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:55.350066 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8943c87d-ed6b-49ca-8115-afa93f6652b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:55.409856 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943c87d-ed6b-49ca-8115-afa93f6652b2 service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:55.413308 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0273abc-82c8-4101-89f5-99edab94d056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c79367697f7451fb804b8641444f6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:55.413978 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0273abc-82c8-4101-89f5-99edab94d056 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:55.414257 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0273abc-82c8-4101-89f5-99edab94d056 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:55.414526 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0273abc-82c8-4101-89f5-99edab94d056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:55.416274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e0273abc-82c8-4101-89f5-99edab94d056 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:55.431249 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0273abc-82c8-4101-89f5-99edab94d056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:55.433896 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0273abc-82c8-4101-89f5-99edab94d056 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:55.453858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1449/2877] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:27:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:27:55.769830 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401ce54-6dd0-4504-912a-31896ce81ddb None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:55.771198 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4401ce54-6dd0-4504-912a-31896ce81ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:55.827451 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401ce54-6dd0-4504-912a-31896ce81ddb None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:55.831061 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9552acb-0b99-4263-be70-1effaf00a05f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:55.831695 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9552acb-0b99-4263-be70-1effaf00a05f service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:55.831976 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9552acb-0b99-4263-be70-1effaf00a05f service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:55.832239 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9552acb-0b99-4263-be70-1effaf00a05f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:55.834728 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c9552acb-0b99-4263-be70-1effaf00a05f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:55.844151 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9552acb-0b99-4263-be70-1effaf00a05f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:55.845561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9552acb-0b99-4263-be70-1effaf00a05f service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:55.853458 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1429/2878] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:27:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:27:56.356921 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0273abc-82c8-4101-89f5-99edab94d056 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:56.359099 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e0273abc-82c8-4101-89f5-99edab94d056 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:56.364379 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9552acb-0b99-4263-be70-1effaf00a05f service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:27:56.376111 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9552acb-0b99-4263-be70-1effaf00a05f service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:56.381474 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1e1cd-d108-46b2-a369-0f90cc921b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:56.382706 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1e1cd-d108-46b2-a369-0f90cc921b31 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:56.383277 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1e1cd-d108-46b2-a369-0f90cc921b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:56.383844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1e1cd-d108-46b2-a369-0f90cc921b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:56.388089 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bdc1e1cd-d108-46b2-a369-0f90cc921b31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:56.411953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc1e1cd-d108-46b2-a369-0f90cc921b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:56.414788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc1e1cd-d108-46b2-a369-0f90cc921b31 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:56.436272 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1430/2879] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:27:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:27:56.442810 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0273abc-82c8-4101-89f5-99edab94d056 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:27:56.448027 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e3b27-0c12-416b-be1b-36075c3e611f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:27:56.448619 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e3b27-0c12-416b-be1b-36075c3e611f service placement] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:27:56.448903 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e3b27-0c12-416b-be1b-36075c3e611f service placement] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:27:56.449176 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e3b27-0c12-416b-be1b-36075c3e611f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:27:56.451139 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5d6e3b27-0c12-416b-be1b-36075c3e611f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:27:56.473565 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e3b27-0c12-416b-be1b-36075c3e611f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:27:56.475427 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e3b27-0c12-416b-be1b-36075c3e611f service placement] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:27:56.486493 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1450/2880] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:27:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:28:06.426171 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=35 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:28:06.481833 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=58 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:28:08.529882 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1e1cd-d108-46b2-a369-0f90cc921b31 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:28:08.539511 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1e1cd-d108-46b2-a369-0f90cc921b31 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:28:08.544170 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409ad21-88f5-4a88-ad4b-eda9572ac534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:28:08.544827 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409ad21-88f5-4a88-ad4b-eda9572ac534 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:28:08.545108 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409ad21-88f5-4a88-ad4b-eda9572ac534 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:28:08.545381 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409ad21-88f5-4a88-ad4b-eda9572ac534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:28:08.547244 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c409ad21-88f5-4a88-ad4b-eda9572ac534 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:28:08.559037 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409ad21-88f5-4a88-ad4b-eda9572ac534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:28:08.560665 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409ad21-88f5-4a88-ad4b-eda9572ac534 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:28:08.571508 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1431/2881] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:28:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:28:11.468516 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e3b27-0c12-416b-be1b-36075c3e611f service placement] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:28:11.479718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e3b27-0c12-416b-be1b-36075c3e611f service placement] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:28:11.483418 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9276bf-202d-4f89-b5cf-2b66ebf17c72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:28:11.484061 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9276bf-202d-4f89-b5cf-2b66ebf17c72 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:28:11.484338 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9276bf-202d-4f89-b5cf-2b66ebf17c72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:28:11.484611 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9276bf-202d-4f89-b5cf-2b66ebf17c72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:28:11.494658 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9276bf-202d-4f89-b5cf-2b66ebf17c72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:28:11.496136 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9276bf-202d-4f89-b5cf-2b66ebf17c72 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:28:11.506523 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1451/2882] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:28:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:28:18.565961 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:28:21.501963 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=6 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:28:53.448423 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409ad21-88f5-4a88-ad4b-eda9572ac534 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:28:53.458822 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409ad21-88f5-4a88-ad4b-eda9572ac534 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:28:53.463258 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff94e6-d867-4f22-939c-edf7677d6afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:28:53.463919 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff94e6-d867-4f22-939c-edf7677d6afd service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:28:53.464327 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff94e6-d867-4f22-939c-edf7677d6afd service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:28:53.464611 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff94e6-d867-4f22-939c-edf7677d6afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:28:53.466526 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-61ff94e6-d867-4f22-939c-edf7677d6afd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:28:53.476769 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff94e6-d867-4f22-939c-edf7677d6afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:28:53.478309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff94e6-d867-4f22-939c-edf7677d6afd service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:28:53.487494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1432/2883] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:28:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:28:54.505436 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9276bf-202d-4f89-b5cf-2b66ebf17c72 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:28:54.519816 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9276bf-202d-4f89-b5cf-2b66ebf17c72 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:28:54.525429 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b1e00-a059-433b-a82d-1b6518cc36d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:28:54.526565 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b1e00-a059-433b-a82d-1b6518cc36d0 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:28:54.526853 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b1e00-a059-433b-a82d-1b6518cc36d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:28:54.527117 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b1e00-a059-433b-a82d-1b6518cc36d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:28:54.528905 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-104b1e00-a059-433b-a82d-1b6518cc36d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:28:54.541915 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b1e00-a059-433b-a82d-1b6518cc36d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:28:54.543850 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b1e00-a059-433b-a82d-1b6518cc36d0 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:28:54.556076 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1452/2884] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:28:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:28:54.593447 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff94e6-d867-4f22-939c-edf7677d6afd service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:28:54.604912 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff94e6-d867-4f22-939c-edf7677d6afd service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:28:54.609219 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd31525-5fd8-48fa-8726-c7e0b316eade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:28:54.609908 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd31525-5fd8-48fa-8726-c7e0b316eade service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:28:54.610241 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd31525-5fd8-48fa-8726-c7e0b316eade service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:28:54.610581 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd31525-5fd8-48fa-8726-c7e0b316eade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:28:54.612394 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7fd31525-5fd8-48fa-8726-c7e0b316eade service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:28:54.627635 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd31525-5fd8-48fa-8726-c7e0b316eade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:28:54.628950 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd31525-5fd8-48fa-8726-c7e0b316eade service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:28:54.640850 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1433/2885] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:28:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:29:04.550991 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=6 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:29:04.635602 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=12 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:30:10.380741 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b1e00-a059-433b-a82d-1b6518cc36d0 service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:30:10.438310 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b1e00-a059-433b-a82d-1b6518cc36d0 service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:30:10.445861 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7553764-3a2e-4c1d-97d0-2b4c66e75ae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:30:10.446747 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7553764-3a2e-4c1d-97d0-2b4c66e75ae3 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:30:10.447031 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7553764-3a2e-4c1d-97d0-2b4c66e75ae3 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:30:10.447301 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7553764-3a2e-4c1d-97d0-2b4c66e75ae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:30:10.449089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e7553764-3a2e-4c1d-97d0-2b4c66e75ae3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:30:10.483373 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7553764-3a2e-4c1d-97d0-2b4c66e75ae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:30:10.485010 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7553764-3a2e-4c1d-97d0-2b4c66e75ae3 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:30:10.496283 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1453/2886] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:30:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:30:11.378032 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a489a60-a586-4b66-978a-be51b36180fa None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:30:11.378539 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a489a60-a586-4b66-978a-be51b36180fa None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:30:11.378743 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a489a60-a586-4b66-978a-be51b36180fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:30:11.397874 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7a489a60-a586-4b66-978a-be51b36180fa None None] MFA Rules not processed for user `7cde3488be6c4c8790841b7529123326`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:30:11.399077 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7a489a60-a586-4b66-978a-be51b36180fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:30:11.406288 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1434/2887] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:30:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:30:11.735658 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7553764-3a2e-4c1d-97d0-2b4c66e75ae3 service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:30:11.754658 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7553764-3a2e-4c1d-97d0-2b4c66e75ae3 service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:30:11.758219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3dfb44-62e1-4cb2-9349-f4210e0479fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:30:11.758848 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3dfb44-62e1-4cb2-9349-f4210e0479fd service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:30:11.759125 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3dfb44-62e1-4cb2-9349-f4210e0479fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:30:11.759394 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3dfb44-62e1-4cb2-9349-f4210e0479fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:30:11.761202 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0f3dfb44-62e1-4cb2-9349-f4210e0479fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:30:11.782803 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3dfb44-62e1-4cb2-9349-f4210e0479fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:30:11.784243 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3dfb44-62e1-4cb2-9349-f4210e0479fd service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:30:11.792817 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1454/2888] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:30:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:30:21.392438 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:30:21.789095 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=46 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:30:36.000561 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a489a60-a586-4b66-978a-be51b36180fa None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:30:36.002854 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7a489a60-a586-4b66-978a-be51b36180fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:30:36.023896 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a489a60-a586-4b66-978a-be51b36180fa None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:30:36.034126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cef460-65cd-48c0-bf56-22a1867a1433 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:30:36.035048 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cef460-65cd-48c0-bf56-22a1867a1433 service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:30:36.035628 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cef460-65cd-48c0-bf56-22a1867a1433 service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:30:36.035903 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cef460-65cd-48c0-bf56-22a1867a1433 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:30:36.038144 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-92cef460-65cd-48c0-bf56-22a1867a1433 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:30:36.058154 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cef460-65cd-48c0-bf56-22a1867a1433 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:30:36.060598 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cef460-65cd-48c0-bf56-22a1867a1433 service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:30:36.073812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1435/2889] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:30:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:30:39.169363 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3dfb44-62e1-4cb2-9349-f4210e0479fd service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:30:39.171014 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0f3dfb44-62e1-4cb2-9349-f4210e0479fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:30:39.300939 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3dfb44-62e1-4cb2-9349-f4210e0479fd service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:30:39.307684 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:30:39.312096 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:30:39.312391 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:30:39.312718 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] PATH_INFO: `/v3/users/73559b7886614dd2a92e9c6c53307331` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:30:39.314544 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] RBAC: Authorizing `identity:delete_user(user_id=73559b7886614dd2a92e9c6c53307331)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:30:39.316654 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:30:39.369035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73559b7886614dd2a92e9c6c53307331'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.378330 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73559b7886614dd2a92e9c6c53307331'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.378954 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73559b7886614dd2a92e9c6c53307331'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.379564 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73559b7886614dd2a92e9c6c53307331'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.394260 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73559b7886614dd2a92e9c6c53307331'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.408505 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1455/2890] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:30:39 2022] DELETE /identity/v3/users/73559b7886614dd2a92e9c6c53307331 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:30:39.598957 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cef460-65cd-48c0-bf56-22a1867a1433 service placement] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:30:39.614508 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cef460-65cd-48c0-bf56-22a1867a1433 service placement] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:30:39.681995 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:30:39.682792 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:30:39.683212 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:30:39.683494 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] PATH_INFO: `/v3/projects/564690a570ea4ec199dc599d288756b0` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:30:39.684844 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] RBAC: Authorizing `identity:delete_project(project_id=564690a570ea4ec199dc599d288756b0)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:30:39.686849 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:30:39.736227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '564690a570ea4ec199dc599d288756b0', 'request_id': 'req-948962b4-3584-41a1-b886-7b484fc29199'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.745452 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '564690a570ea4ec199dc599d288756b0', 'request_id': 'req-948962b4-3584-41a1-b886-7b484fc29199'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.752251 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '564690a570ea4ec199dc599d288756b0', 'request_id': 'req-948962b4-3584-41a1-b886-7b484fc29199'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.763788 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] The token cache is being invalidate because project 564690a570ea4ec199dc599d288756b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:30:39.764211 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.764886 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.766530 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1436/2891] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:30:39 2022] DELETE /identity/v3/projects/564690a570ea4ec199dc599d288756b0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:30:39.770538 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:30:39.772640 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:30:39.786167 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5abb9-a085-425d-97a1-c577f04ac24b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:30:39.837481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:30:39.849913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:30:39.851352 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:30:39.851649 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:30:39.852168 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] PATH_INFO: `/v3/users/8ac84f3961484c6dba931ff12a24e48c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:30:39.853821 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ac84f3961484c6dba931ff12a24e48c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:30:39.855757 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:30:39.908319 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ac84f3961484c6dba931ff12a24e48c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.916434 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ac84f3961484c6dba931ff12a24e48c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.917169 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ac84f3961484c6dba931ff12a24e48c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.917919 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ac84f3961484c6dba931ff12a24e48c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.931208 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ac84f3961484c6dba931ff12a24e48c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:39.944904 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1456/2892] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:30:39 2022] DELETE /identity/v3/users/8ac84f3961484c6dba931ff12a24e48c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:30:40.132813 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:30:40.135062 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:30:40.153069 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948962b4-3584-41a1-b886-7b484fc29199 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:30:40.219714 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7126af-5c98-4599-a9b4-10d597072273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:30:40.220615 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7126af-5c98-4599-a9b4-10d597072273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:30:40.220970 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7126af-5c98-4599-a9b4-10d597072273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:30:40.221309 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7126af-5c98-4599-a9b4-10d597072273 admin admin] PATH_INFO: `/v3/projects/8623d691c6384d53bb1fd6c9ecc3763f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:30:40.222637 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7126af-5c98-4599-a9b4-10d597072273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8623d691c6384d53bb1fd6c9ecc3763f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:30:40.225227 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7126af-5c98-4599-a9b4-10d597072273 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:30:40.248129 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:30:40.276915 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8f7126af-5c98-4599-a9b4-10d597072273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8623d691c6384d53bb1fd6c9ecc3763f', 'request_id': 'req-8f7126af-5c98-4599-a9b4-10d597072273'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:40.285248 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8f7126af-5c98-4599-a9b4-10d597072273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8623d691c6384d53bb1fd6c9ecc3763f', 'request_id': 'req-8f7126af-5c98-4599-a9b4-10d597072273'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:40.292112 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8f7126af-5c98-4599-a9b4-10d597072273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8623d691c6384d53bb1fd6c9ecc3763f', 'request_id': 'req-8f7126af-5c98-4599-a9b4-10d597072273'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:40.303889 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8f7126af-5c98-4599-a9b4-10d597072273 admin admin] The token cache is being invalidate because project 8623d691c6384d53bb1fd6c9ecc3763f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:30:40.304179 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8f7126af-5c98-4599-a9b4-10d597072273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:40.305020 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8f7126af-5c98-4599-a9b4-10d597072273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:30:40.306725 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1437/2893] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:30:40 2022] DELETE /identity/v3/projects/8623d691c6384d53bb1fd6c9ecc3763f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:30:49.940441 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:30:50.299946 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:30:54.339875 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:30:54.341280 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:30:54.432657 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c094de-1e41-422d-94b8-9e17c8d596c7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:30:54.491922 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d2c01-ca42-4191-8556-bad05a0d7f8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:30:54.492571 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d2c01-ca42-4191-8556-bad05a0d7f8a service placement] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:30:54.492851 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d2c01-ca42-4191-8556-bad05a0d7f8a service placement] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:30:54.493122 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d2c01-ca42-4191-8556-bad05a0d7f8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:30:54.494901 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f22d2c01-ca42-4191-8556-bad05a0d7f8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:30:54.505895 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d2c01-ca42-4191-8556-bad05a0d7f8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:30:54.508285 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d2c01-ca42-4191-8556-bad05a0d7f8a service placement] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:30:54.518082 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1457/2894] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:30:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:31:04.513811 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=67 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:32:04.968634 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84b2e1-75b5-4b68-93d1-195c51a2b012 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:32:04.969177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84b2e1-75b5-4b68-93d1-195c51a2b012 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:32:04.969177 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84b2e1-75b5-4b68-93d1-195c51a2b012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:32:04.987715 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ae84b2e1-75b5-4b68-93d1-195c51a2b012 None None] MFA Rules not processed for user `7cde3488be6c4c8790841b7529123326`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:32:04.988906 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ae84b2e1-75b5-4b68-93d1-195c51a2b012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:32:05.043965 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1438/2895] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 18:32:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:32:15.036521 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=25 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:32:23.179971 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d2c01-ca42-4191-8556-bad05a0d7f8a service placement] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:32:23.182730 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f22d2c01-ca42-4191-8556-bad05a0d7f8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:32:23.281641 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d2c01-ca42-4191-8556-bad05a0d7f8a service placement] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:32:23.287033 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364d5a1-e320-45ea-b6b5-402ef3c72668 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd420e808730d484f80963322c5fc6856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:32:23.287704 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364d5a1-e320-45ea-b6b5-402ef3c72668 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:32:23.287984 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364d5a1-e320-45ea-b6b5-402ef3c72668 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:32:23.288256 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364d5a1-e320-45ea-b6b5-402ef3c72668 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:32:23.290815 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2364d5a1-e320-45ea-b6b5-402ef3c72668 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:32:23.303029 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364d5a1-e320-45ea-b6b5-402ef3c72668 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:32:23.304858 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364d5a1-e320-45ea-b6b5-402ef3c72668 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:32:23.318370 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1458/2896] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:32:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:32:33.311625 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=43 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:33:55.107571 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84b2e1-75b5-4b68-93d1-195c51a2b012 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:33:55.110175 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ae84b2e1-75b5-4b68-93d1-195c51a2b012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:33:55.176542 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84b2e1-75b5-4b68-93d1-195c51a2b012 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:33:55.180762 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd2f36-49ed-440b-8735-6cc43d2f356d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:33:55.181446 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd2f36-49ed-440b-8735-6cc43d2f356d service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:33:55.181740 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd2f36-49ed-440b-8735-6cc43d2f356d service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:33:55.182047 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd2f36-49ed-440b-8735-6cc43d2f356d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:33:55.189255 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e9cd2f36-49ed-440b-8735-6cc43d2f356d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:33:55.200812 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd2f36-49ed-440b-8735-6cc43d2f356d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:33:55.203247 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd2f36-49ed-440b-8735-6cc43d2f356d service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:33:55.212579 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1439/2897] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:33:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:34:01.784938 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364d5a1-e320-45ea-b6b5-402ef3c72668 service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:34:01.794522 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364d5a1-e320-45ea-b6b5-402ef3c72668 service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:34:01.798237 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18949e92-6141-47e5-a6d8-d3afac4accd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:34:01.798902 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18949e92-6141-47e5-a6d8-d3afac4accd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:34:01.799183 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18949e92-6141-47e5-a6d8-d3afac4accd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:34:01.799481 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18949e92-6141-47e5-a6d8-d3afac4accd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:34:01.801344 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-18949e92-6141-47e5-a6d8-d3afac4accd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:34:01.813133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18949e92-6141-47e5-a6d8-d3afac4accd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:34:01.814602 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18949e92-6141-47e5-a6d8-d3afac4accd3 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:34:01.824773 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1459/2898] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:34:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:34:02.885245 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd2f36-49ed-440b-8735-6cc43d2f356d service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:34:02.899074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd2f36-49ed-440b-8735-6cc43d2f356d service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:34:02.909435 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bf8b0-b1b9-48f4-84f9-1fe1b81479c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:34:02.909926 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bf8b0-b1b9-48f4-84f9-1fe1b81479c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:34:02.910760 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bf8b0-b1b9-48f4-84f9-1fe1b81479c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:34:02.911034 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bf8b0-b1b9-48f4-84f9-1fe1b81479c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:34:02.915097 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-025bf8b0-b1b9-48f4-84f9-1fe1b81479c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:34:02.945243 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bf8b0-b1b9-48f4-84f9-1fe1b81479c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:34:02.947121 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bf8b0-b1b9-48f4-84f9-1fe1b81479c9 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:34:02.963861 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1440/2899] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:34:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:34:04.180112 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18949e92-6141-47e5-a6d8-d3afac4accd3 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:34:04.190717 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18949e92-6141-47e5-a6d8-d3afac4accd3 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:34:04.195064 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25ee39-0220-4d30-880c-f88dbd477740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:34:04.195703 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25ee39-0220-4d30-880c-f88dbd477740 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:34:04.195980 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25ee39-0220-4d30-880c-f88dbd477740 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:34:04.196248 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25ee39-0220-4d30-880c-f88dbd477740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:34:04.198035 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-eb25ee39-0220-4d30-880c-f88dbd477740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:34:04.213021 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25ee39-0220-4d30-880c-f88dbd477740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:34:04.214511 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25ee39-0220-4d30-880c-f88dbd477740 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:34:04.229098 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1460/2900] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:34:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:34:12.955588 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=32 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:34:14.221530 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=12 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:34:14.596463 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bf8b0-b1b9-48f4-84f9-1fe1b81479c9 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:34:14.605465 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bf8b0-b1b9-48f4-84f9-1fe1b81479c9 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:34:14.610029 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddc859-3753-47ab-9217-a8f726e04035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f668c4c6745e4f1eae20d8a3a12cf0cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:34:14.610672 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddc859-3753-47ab-9217-a8f726e04035 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:34:14.610953 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddc859-3753-47ab-9217-a8f726e04035 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:34:14.611225 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddc859-3753-47ab-9217-a8f726e04035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:34:14.613074 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-70ddc859-3753-47ab-9217-a8f726e04035 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:34:14.682222 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddc859-3753-47ab-9217-a8f726e04035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:34:14.683673 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddc859-3753-47ab-9217-a8f726e04035 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:34:14.693989 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1441/2901] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:34:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:34:21.031591 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25ee39-0220-4d30-880c-f88dbd477740 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:34:21.033089 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-eb25ee39-0220-4d30-880c-f88dbd477740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:34:21.046615 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25ee39-0220-4d30-880c-f88dbd477740 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:34:21.052044 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:34:21.052910 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:34:21.053185 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:34:21.053453 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] PATH_INFO: `/v3/users/7cde3488be6c4c8790841b7529123326` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:34:21.054983 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cde3488be6c4c8790841b7529123326)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:34:21.056856 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:34:21.106723 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cde3488be6c4c8790841b7529123326'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.115312 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cde3488be6c4c8790841b7529123326'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.115930 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cde3488be6c4c8790841b7529123326'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.116537 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cde3488be6c4c8790841b7529123326'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.129133 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cde3488be6c4c8790841b7529123326'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.142233 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1461/2902] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:34:21 2022] DELETE /identity/v3/users/7cde3488be6c4c8790841b7529123326 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:34:21.321070 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddc859-3753-47ab-9217-a8f726e04035 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:34:21.332203 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddc859-3753-47ab-9217-a8f726e04035 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:34:21.392403 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:34:21.393268 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:34:21.393550 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:34:21.393852 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] PATH_INFO: `/v3/projects/6774c3ce4b094e19ad6da95e3744d9ed` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:34:21.395143 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6774c3ce4b094e19ad6da95e3744d9ed)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:34:21.397083 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:34:21.444828 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6774c3ce4b094e19ad6da95e3744d9ed', 'request_id': 'req-3362383a-0df7-4c32-985a-2b24395f2592'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.452126 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6774c3ce4b094e19ad6da95e3744d9ed', 'request_id': 'req-3362383a-0df7-4c32-985a-2b24395f2592'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.459808 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6774c3ce4b094e19ad6da95e3744d9ed', 'request_id': 'req-3362383a-0df7-4c32-985a-2b24395f2592'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.470754 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] The token cache is being invalidate because project 6774c3ce4b094e19ad6da95e3744d9ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:34:21.471039 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.471679 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.473220 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1442/2903] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:34:21 2022] DELETE /identity/v3/projects/6774c3ce4b094e19ad6da95e3744d9ed => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:34:21.477420 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:34:21.478972 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:34:21.492219 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa7df4-dc85-4392-b9a2-b7ca91ddbdb1 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:34:21.549605 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:34:21.550500 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:34:21.550781 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:34:21.551049 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] PATH_INFO: `/v3/users/36a8414532e24fc9badf492770dd75af` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:34:21.552225 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] RBAC: Authorizing `identity:delete_user(user_id=36a8414532e24fc9badf492770dd75af)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:34:21.554458 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:34:21.605772 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36a8414532e24fc9badf492770dd75af'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.618207 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36a8414532e24fc9badf492770dd75af'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.619102 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36a8414532e24fc9badf492770dd75af'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.619964 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36a8414532e24fc9badf492770dd75af'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.632687 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36a8414532e24fc9badf492770dd75af'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:21.634630 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:34:21.648250 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1462/2904] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 18:34:21 2022] DELETE /identity/v3/users/36a8414532e24fc9badf492770dd75af => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:34:22.083426 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:34:22.084871 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:34:22.100256 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362383a-0df7-4c32-985a-2b24395f2592 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:34:22.122773 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:34:22.163302 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab5447-29f3-4c1f-9417-1332b936bf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b98d845867d491c91aeca1e12cf3ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbbdb1cacacf4ba5a41a00d6d478efbe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:34:22.164148 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab5447-29f3-4c1f-9417-1332b936bf02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:34:22.164429 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab5447-29f3-4c1f-9417-1332b936bf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:34:22.164702 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab5447-29f3-4c1f-9417-1332b936bf02 admin admin] PATH_INFO: `/v3/projects/1ca302a96a414709a24f15fd03edc0d6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:34:22.165890 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab5447-29f3-4c1f-9417-1332b936bf02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ca302a96a414709a24f15fd03edc0d6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:34:22.167798 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab5447-29f3-4c1f-9417-1332b936bf02 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:34:22.214469 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d3ab5447-29f3-4c1f-9417-1332b936bf02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ca302a96a414709a24f15fd03edc0d6', 'request_id': 'req-d3ab5447-29f3-4c1f-9417-1332b936bf02'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:22.223184 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d3ab5447-29f3-4c1f-9417-1332b936bf02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ca302a96a414709a24f15fd03edc0d6', 'request_id': 'req-d3ab5447-29f3-4c1f-9417-1332b936bf02'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:22.229614 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d3ab5447-29f3-4c1f-9417-1332b936bf02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ca302a96a414709a24f15fd03edc0d6', 'request_id': 'req-d3ab5447-29f3-4c1f-9417-1332b936bf02'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:22.240147 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d3ab5447-29f3-4c1f-9417-1332b936bf02 admin admin] The token cache is being invalidate because project 1ca302a96a414709a24f15fd03edc0d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:34:22.240431 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d3ab5447-29f3-4c1f-9417-1332b936bf02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:22.241081 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d3ab5447-29f3-4c1f-9417-1332b936bf02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:34:22.242676 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1443/2905] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 18:34:22 2022] DELETE /identity/v3/projects/1ca302a96a414709a24f15fd03edc0d6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:34:31.645703 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:34:32.237043 dv-cinder-843493-1-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:35:58.338744 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:35:58.340758 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:35:58.447823 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da4155-9907-493d-8423-d1dbc972a18b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:35:58.528628 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e99c07-7ad4-48c2-8c1b-747fcb8be9d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ee40bcb7905490c9ad3e39e7a93b122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4be4045293754773bb52bbb82546a1db', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:35:58.529347 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e99c07-7ad4-48c2-8c1b-747fcb8be9d4 service placement] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:35:58.529633 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e99c07-7ad4-48c2-8c1b-747fcb8be9d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:35:58.529959 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e99c07-7ad4-48c2-8c1b-747fcb8be9d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:35:58.531913 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-33e99c07-7ad4-48c2-8c1b-747fcb8be9d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:35:58.545485 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e99c07-7ad4-48c2-8c1b-747fcb8be9d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:35:58.547272 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e99c07-7ad4-48c2-8c1b-747fcb8be9d4 service placement] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:35:58.559274 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1463/2906] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 18:35:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:36:08.554336 dv-cinder-843493-1-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=58 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}